1. Why MetaFactory?

Once upon a time (in 2004) Marnix Bochove, then working as a senior software developer at a large financial institution, realized that very often he was programming the same kind of code over and over again. So he began to investigate the possibilities of approaching software production just like production in the manufacturing world. Hence the name of the current company: MetaFactory. The aim was to create a software production facility, far more versatile than a common software generator. Most generators can generate rather standard solutions, but often they create ugly code. The idea was that the developer should keep total control over the produced code which should be of eminent quality.

The years went by. Now there is the MetaFactory Composer. A highly versatile production environment which you can integrate into your development processes to an extend that you can choose yourself. It can generate source code in different languages and is independent of any framework.

Every software system bigger than ‘Hello World’ contains pieces of rather generic logic that occur repeatedly in the sources. These pieces are called ‘patterns’. Using the MetaFactory Composer you once create a code instruction for a pattern. Then you can generate all occurrences of the pattern with a push on a button. You might expect that all software developers are yearning for a piece of equipment like this.

But they don’t.

I worked with many engineers, man and women: electrical, mechanical, hydraulical, software, maintenance and documentation engineers. They appear to have one thing in common: a string of Gyro Gearloose’s (in Dutch: Willie Wortel) genes in their DNA. Deep down they want to invent things. And find the perfect solution. But after having found the perfect way, circumstances change (for instance by change requests). And a new quest for the best solution can begin.

The old solutions are buried in the code already created. And remain there until major issues force the engineer to update the deprecated code. And this is what software engineers hate (I know because I’ve been one myself): digging through a multitude of (somebody else’s) source files to get the damned thing working again.

Most software engineers are not eager to leave their cosy environment with their own language, frameworks, programming style and habits. But MetaFactory offers opportunities to improve programming continuously, at a high productivity level and to be more flexible towards change requests and technology changes.

This story is about why Metafactory can be your next step in the professionalization of software engineering.

1.1. What do Developers need?

Each year the Standish Group https://www.standishgroup.com publishes the results of a survey on how succesful ICT projects are. According to interviewed ICT managers the most important success factors for a software project are:

• Small projects,

• Strong user commitment.

• Strong commitment of higher management.

• Experienced project manager.

• Clear Business Case.

We know that already for -say- 35 years, but still about 50% of the software projects fail. So there must be other issues that are overlooked.

In their Developers Survey of 2016 (49.521 respondents worldwide) Stack Overflow asked What are your challenges at work?.

Fig. 1.1 Results Developers Survey of 2016 (partly)

It is likely that the efficiency of development processes will deteriorate due to poor documentation, a fragile code base and outdated technologies. And, if the development processes are less efficient than expected, this will lead to unrealistic expectations.

So what developers need is:

Efficient Development Processes, with tooling that:

• Enhances development speed,

• Is flexible towards changing requirements,

• Can easily cope with technology changes,

• Fortifies the Code base and

• Supports documentation.

1.2. The conventional way of development

Fig. 1.2 Conventional development

Looking at Fig. 1.2 we see the following:

• Already at the first modification of the blue file, a bug infects the code. This stays existent in all consequent copies.

• At the test all sources must be checked for the bug.

• After a change request all sources (except black, because the change is not relevant there) must be modified.

• Unluckily new bugs are intruduced during modification.

• During the modification from red to black, the developer had a brilliant idea. The implementation of this new insight stays restricted to the black file. The other sources do not benefit from the new insight. As a result the implementations grow apart.

The consequences are:

• Logic (and bugs) end up scattered over a multitude of source files and are frozen there.

• During maintenance there’s a lot of (repeating) work in many files.

• The process is sensitive for differences in style and interpretation and

• Sensitive for changes.

• Progressive insight is buried in individual files.

1.3. Automating the software production

Just like the manufacturing industry:

• Avoid duplication of work.

• Automate repeating or generic ‘patterns’.

Patterns, where are they?

Fig. 1.4 Patterns in two projects

In Fig. 1.4 we see two systems, connected to the internet via a WEBserver. Both make use of a database. In the system on top, there live three objects: oranje cap, black car and tree. Each of them needs screen handling functions, a REST interface, business ligic, and an interface to the backend. These 4 we call ‘patterns’. Except for the business logic, which will be very specific, the other 3 (screen handling, REST interface and backend coupling) are applicable to all objects (entities) in the system.

And when there is another system with similar objects, as on the bottom of the figure, the same patterns are applicable there too.

1.4. Automation, in which case?

We make a distiction between the needs of an individual user who needs automation for his work processes and the needs of a professional software developer.

Fig. 1.5 User focus on project parameters

The individual user is not interested in code or code quality. He or she is primarily focussed on quick results and the ability to make rapid modifications when questions or requests change. And is not inclined to write many lines of source code. If you want to support this user with automation, a Low Code Solution might be a good option.

The professional developer’s job however is to create source code. And to keep control over it.

Fig. 1.6 Developer focus

In general he or she will be involved in larger projects where the importance of maintainability and code quality grow with increasing lifespan of the project’s endproduct.

If a project’s size is relatively small (little repetition, not too many lines of code) and the demands for code quality and maintainability are moderate, the conventional way of manual programming may suit well. But if the rate of repetition and the demands for high productivity, high code quality and maintainability are higher, the effort to keep control over the quality (bugfixing, consistency, programming style) of a large number of sources increases rapidly. Then conventional programming is not a workable method any more.

Then automation with the MetaFactory Composer is the better way. It eliminates repetition, creates high productivity, supports continuous improvement of the code and keeps the code uniform and structured.

1.5. MetaFactory Composer, what is it?

The Composer is a source code production environment. It can create:

• Java

• Typescript

• XML

• HTML

The product is source code. Not more, not less. The produced code can be combined with any other code.

So the Composer and its output can blend in with any existing development process. If you like it can take control over the whole process. But if you don’t, you can restrict it to the production of a few files. In any case you’re still the owner of the code and you keep for 100% control over it. You can keep using the frameworks that you are comfortable with.

1.6. Working Principle

1. During design a number of objects, or entities, is defined. The objects are described in a model.

2. MetaFactory generates source code for the objects, based on Code Instructions. These are sets of instructions, which control the generation process. A single Code Instruction can be applied to many objects.

3. The source code must be compiled, just like in the conventional way.

4. With a special Code Instruction the Composer can, based on the model, create a database generation script. This makes generation of a new database as ‘easy as a breeze’.

5. The same is true for testing. If a suitable Code Instruction is available, the Composer can create test scripts.

This technique opens the door for a lot of flexibility:

• When an object has changed, source code and database can be generated in one run. The code and the database always stay synchronized. And the test scripts are updated too.

• If you change a Code Instruction, all sources, based on that Code Instruction are updated. No make-over for all individual sources any more.

There are some standard Code Instructions available, but you are not obliged to use them. Over time you will create your own collection of Code Instructions which will enhance your production process considerably.

1.7. Create Code Instruction

1. Start with hand written code. It may be manually coded based on your design, copied, stolen or whatever.

2. Identify the logic that is suitable for use in general situations. Don’t bother about the details and specifics, that’s for later. Create a Code Instruction, based on that general logic.

3. Run the generation process.

4. Examine and evaluate the generated code.

5. If the results are not as expected, modify the Code Instruction and run the generation again until you are safisfied with the results.

Iteration does it

This model offers the possibility of iteration for different reasons:

• Initially

  • Write Code Instruction for the generation of the right source code.

  • Iterate until it is (almost) perfect.

• Changes

  • In the Model: Change the Model and generate the code again.

  • In functionality: Modify one or more Code Instructions and generate again.

• Progressive insight

  • Modify one or more Code Instruction and generate again.

Here’s the real power:

• Quality improves at each iteration

  • Is saved in Code Instructions and

  • Applied to all generated sources.

• Progressive Insight

  • Is saved in Code Instructions and

  • Applied to all generated sources.

• Best practices

  • Are collected in Code Instructions

  • Earlier produced code profits too

1.8. Diversify

The situation depicted in Section 1.6 is rather simple. With a plain Code Instruction all generated sources will be similar, except for some object-bound matters. For most projects this will not be sufficient.

But don’t worry. Our toolbox is far from empty.

1. Metadata

   You can enrich the model with metadata.

   Metadata control the generation process of the Code Instruction and make object dependent generation of code possible. So with different metadata, the same Code Instruction can generate different stuff.

2. Snippet

   A Code Instruction is written in XML, which is not a programming language and lacks all the fancy constructs a programmer may need. However, these programming facilities can be injected into a Code Instruction in the form of one or more snippets. With snippets a Code Instruction can generate all the generic (say: default) functions for objects.

3. Hook

   But there may be a need for project-specific behaviour. That’s what hooks are for.

   They are project dependent snippets, are called by a snippet and can add extra functionality to or change the behaviour of the snippet. In general a hook modifies the output of the Code Instruction.

4. Feature

   A feature is functionality as a result of a combination of Code Instructions.

1.9. Reuse

Whether you are developing for a car dealer, a bookstore, a tools rental or for a financial service, every time you are confronted with a numer of objects or entities with characteristics that must be stored in a database and be maintained. These objects have a lot in common, across projects, across companies, even across branches. Normaly the common stuff has to be coded over and over again for each object. The Composer however can, with the same Code Instruction, generate code for many objects.

A Code Instruction, once built, can be used over and over again. This is the real strength of the Composer.

1. Well designed and well built Code Instructions can be collected in libraries.

   MetaFactory is developing standard libraries with basic Code Instructions which are useful for almost every project.

2. Reuse

   For a specific project you can create specific Code Instructions. But the real benefit is in using general Code Instructions, assisted by hooks for project specificy.

1.10. The Big Benefit

In every project you will find patterns.

Fig. 1.12 Patterns in a project

In Fig. 1.12 we see 3 objects (cap, car, tree) and 4 patterns (screen, database, Rest and egg)

Fig. 1.13 4 Code Instructions and 1 Model. Output: 12 files with source code.

With a Code instruction for every pattern the Composer will generate 12 sources: 4 for each object.

For project specific changes in a new project, hooks can be applied.

1.11. So what’s in it for me, as a developer?

• The need for your specific expertise (Java, TypeScript, frontend, backend, architecture) will only grow.

• Your knowledge and personal development grow along with the Best Practices.

• You will spend more time on analysis and composing, less on coding.

• It becomes easier to adopt new technologies.

• You get more appreciation because you deliver high quality code in a short time and you are flexible in honouring change requests.

• Using the Composer you can deliver the whole project consistent with the latest insights.