【Microservice Theory】01. Overview of Microservices

Why learn architecture?

Because architectural ideas are the cornerstone of coding, if you don’t have architectural ideas in your mind, coding will be blind, and it is impossible to write good code without seeing the overall situation.

Don't rush the code, first look at the theory, there is a general concept. The principle may be understood by looking at it, but only after experiencing the practice and experiencing the pain points can it be understood and absorbed.

Architecture selection

The way of organization and communication determines the system design.

You must choose the architecture according to your company's size and business form. A good architecture evolves step by step. Don't advocate new technologies and new architectures too much.

The evolution of the architecture must come from pain points.

If you don't know what is wrong with the current architecture and what are the pain points, then it must not evolve. Many companies or individuals see that microservices are very popular and start blindly pursuing them. This can only be the reason for seeking fish.

With regard to microservices, I can only say that there is no whole body in a single challenge.

The first problem of microservices:

• Should we introduce microservices
• What technologies are needed for the microservice system
• How does the team evolve and how to land

This series of articles published some personal humble opinions on these issues, and I hope they can be helpful and welcome suggestions from the big guys.

Monolithic architecture, monolithic architecture

Before I talk about microservices, I want you to introduce the monolithic application. If you don't know the pain of monolithic applications, you won't have a deep understanding of the value of microservices.

When the company team is small, a single application is the best choice. The main modes are: LANMP (Linux + Apache or Nginx + MySQL + PHP) and MVC (Spring + iBatis/Hibernate + Tomcat). Its advantages are low cost of learning, quick start to development, convenient testing, deployment, and operation and maintenance, and even one person can complete the development and deployment of a website.

The first choice for a small team of 1-4 people is to let go of your arms.

Don't just come up with microservices. A small company with a few developers pursues microservices and mid-office architecture. Is this the pursuit of perfection? No, it's looking for death.

I have time later to write an article about Conway's Law .

Advantages of monolithic architecture

• Fast, in addition to fast or fast, they are all together, the function is called directly, why is it not fast, the logs are also together, what causes the whole body to move, I am the whole body.

Disadvantages of monolithic architecture

• The applications are complex and all coupled together.
  • This leads to difficulty in refactoring, difficulty in agile development, and collapse in one fell swoop. Move the whole body by pulling a hair.
  • It is difficult to expand, and no one will dare to move after a long time. New colleagues need to read a lot of code when they come, and the learning cost is extremely high. If the relevant person in charge leaves, it will be an avalanche effect.
• IDE card, code are all together, of course the card.
• Slow compilation and slow startup. When monolithic applications have more and more codes and rely on more and more resources, the application can be compiled, packaged, deployed and tested once, and it may even take more than 10 minutes.
• It is difficult to upgrade the framework. You can only upgrade to full upgrades, and you cannot use language advantages in a targeted manner.
• Team development costs are high. In my experience, in the early days when there were only two or three developers in the team, the code was modified collaboratively, and finally merged into the same master branch, and then packaged and deployed, which was still controllable. But once the team expands, more than 5 people modify the code, and then package and deploy together. As long as there is a problem with a function in the test phase, you have to recompile the package and deploy, and then preview the test again. All relevant developers have to participate in it again, efficiency Low and extremely high development cost.

Microservice 1.0

If something is too big, the best solution is to make it smaller.

Divide and conquer, turning the complex into simplicity.

In my opinion, microservices are also hierarchical. That is the degree of micro.

For example, it can be divided into big logic first, using e-commerce as an example, product display, purchase, and logistics are divided into this way.
It can also be divided into commodity services, order services, payment services, etc.

What is the difference?
The former is just divided by interface . I put all the displayed interfaces in a single unit to write, so that the single unit hangs and the purchaser will not be affected.
The problem is that I have to write multiple sets of public modules (authentication, low-level methods).
Suitable for medium-sized teams, because public modules are not changed every day.

The business coupling is reduced to the greatest extent. Module A will not affect Module B if it is hung up, and modules can be assigned to different people and managed separately.
It’s troublesome offline, but stable online.

For example: account module. At this time, there may be a set of account data generation codes in each unit, and then maintain it with the same git. Once updated, all units must be updated.

Although it is also a modular logic, it will eventually be packaged and deployed as a monolithic application.

problem

• The code redundancy is serious.
• If a single machine cannot be carried, only load balancing can be used. Checking logs may require checking multiple machines. Updates are also required to upload codes in bulk.
• Databases are difficult to isolate, especially for users with the largest amount of data. A bug may cause overall slowness.
• The failure cannot be transferred. The A machine hangs up, and the code of the B machine is the same, and it basically hangs too.
• The impact is on the entire business, which cannot be downgraded or fused. Hanging is a business hanging.

Microservice 2.0

The other is to divide by module , or divide by data. This is the real microservice. SOA (Service Oriented Architecture Model) is
suitable for large companies with large business volumes, and each module is responsible for a small team.

Minimize communication costs.

When the number of personnel comes up, the biggest consumption is not coding time, but communication time.

Example: I centralize my account to one service, and everyone who needs account information asks me to get it. Then I use load balancing and clustering to ensure my high availability. If someone wants to update user data, I need to use my interface. Either go to my message queue and I will maintain the consistency of account data.

Built around business or data, services focus on a single business. A lightweight communication mechanism is adopted between services, which can be deployed fully automatically and independently, and can use different programming languages ​​and data storage technologies. The microservice architecture realizes service componentization through business splitting, and quickly develops the system through component combination, and the single service component of the business can be deployed independently, making the entire system clear and flexible.

• Small is beautiful : small service codes are less, bugs are less, easy to test, easy to maintain, and it is easier to iterate continuously to perfect the exquisite and beautiful.
• Single responsibility : a service only needs to do one thing well, focus can do well.
  • One service only does one thing
  • One bag does only one thing
  • A function does only one thing
• Create prototypes as early as possible: Provide service APIs as early as possible, establish service contracts, and reach a consistent agreement for communication between services. As for implementation and improvement, you can do it slowly.
• Portability is more important than efficiency : The light-weight interaction protocol between services still considers compatibility and portability between efficiency and portability.
  

what is the problem?

Fred Brooks wrote 30 years ago,

“there are no silver bullets”。

But everything has advantages and disadvantages, and microservices are not a panacea.

• Natural complexity
  • Will there be bugs in the calls between each module?
  • Module A adjusts B, C, D, and there is a problem in one link, do I have to look at the error log of each module?
  • How to ensure data consistency?
• The cost of communication may even rise in the short term.
  • It was originally a local method of mine, but now I want to convert it to remote call, interface or RPC, do I have to judge whether ping works or not.
  • Originally, I could directly see the logic in the code. Now if there is a problem with the call, I want to ask, and then the provider has to add documentation.
• Difficult to test
  • It turns out that the test is over as soon as the single application is updated. Now that the code of module B is updated, it is incompatible or has bugs. Is it because the modules that depend on him are completely broken?
• Avoid chain reactions . The upstream (caller) writes a for loop to the downstream (provider), and then downstream to his downstream, which may be exponentially magnified.
• Compatibility issues . Once everyone is different, there are bound to be many compatibility issues.
• The dependency is complicated . Perhaps in the end, the interdependence between services will become a mess.
• Distributed transaction issues .

How to deal with it?

• The infrastructure is set up: message queue, log collection, container arrangement, etc.
• The document is written, and the gRPC code is the document.
• Automatic deployment, CICD: Gitlab + Gitlab Hooks + k8s
• Dyeing release is convenient for development and pressure testing.
• Monitoring system: k8s, and a series of Prometheus, ELK, Conrtol Panle
• Testing: testing environment, unit testing, API automation testing

benefit

Since so many people choose microservices, the advantages must outweigh the disadvantages.

• Very low coupling. (Decentralization)
  • Data decentralization : Exclusive use of DB to reduce data interference. In the past, there would not be a situation where a slow SQL hit the whole server.
  • Technology decentralization : You can use any language you want, as long as you implement the protocol. Facilitate reconstruction and iteration.
  • Decentralization of governance :
    • Independent process, isolated deployment. Targeted clusters will add machines for hotspot services.
    • Monitoring and logs are all together, and problems can be found quickly.
• You can use parallel thinking to call data, which is faster than serial.
• Reduced communication costs, everyone can maintain their own services.
• After the split, the amount of code is less, and bugs are easier to find.

Implement microservices

• kit: A basic library (framework) for microservices.
• service: business microservices composed of business code + kit dependency + third-party dependency
• rpc + message queue: lightweight communication

Essentially equivalent to multiple microservices compose (compose) to complete a complete user scenario (usecase).

Availability

This is a big topic, and I will talk about it slowly later.

• isolation
• Timeout control
• Load protection
• Limiting
• Downgrade
• Retry
• Load balancing

compatibility

Be conservative in what you send, be liberal in what you accept.

Be conservative when sending and open when receiving.
When designing and implementing services according to Burstal’s law, the data sent should be more conservative, which means that the necessary information should be transmitted to a minimum, and more open when receiving means to tolerate redundant data to the greatest extent and ensure compatibility. .

Everything that depends on will fall apart.
Everything that depends on will fall apart.
Everything that depends on will fall apart.

to sum up

There is no best architecture, only the one that suits you best. Customize the architecture according to your own business and team, and an architecture that can be delivered quickly, iterated quickly, and continuously refactored is a good architecture.