However, the author encountered some serious problems after the project was actually launched :
1. Lack of complete API documentation and specifications :
· The API documentation is incomplete and full of errors and cannot be used as an accurate reference.
· There is no agreed upon API testing process, allowing the backend development team to change and release the API at will without notice. Such changes are often bugs and break integration with API clients.
2. Inappropriate API framework and validation :
The backend team used an inappropriate Flask framework and implemented complex payload validation on their own, most of which was not tested.
· The verification part has a large amount of code, including date, timestamp and string format verification, but it has not been fully tested, which seriously slows down the project progress.
Therefore , to solve the above problems, it is necessary to repair and improve the API documentation to ensure its accuracy and completeness ; establish an appropriate API framework to improve code quality and testability ; and establish strict API specifications to prevent the release of versions that do not meet the specifications.
Fix API design and documentation
It can be seen from the problems exposed above that the development team does not understand the best practices of REST API and the OpenAPI specification. Therefore, the development team needs to first understand OpenAPI and how it works, and integrate the API documentation into the OpenAPI specification to have a clearer understanding of the expected behavior of the API.
In the process of integrating documentation into the OpenAPI specification, we discovered many problems with previous API designs. For example, custom date formats previously required custom validation logic in the server and UI. We solved this problem by switching to the ISO standard supported by OpenAPI to represent dates.
In addition, some modes are so flexible that validation is almost ineffective. To improve validation, we refactored the schemas and created a model with different endpoints for each entity, making it more usable.
HTTP methods and status codes were used inappropriately in the previous design. The team only uses GET and POST methods, and all responses return a 200 status code. This improper use makes it unclear when trying to create, delete, or update resources. To solve this problem, we have redefined the use of HTTP methods and correctly return the appropriate status code to more accurately indicate the success or failure of the request.
Another limitation of the previous design was "endpoint reuse". Although it seems to save code, it actually exposes too many implementation details. Therefore, we must emphasize the concept of designing the API first and then considering the implementation.
Merging the API specification with OpenAPI was an important turning point for the project. Thereafter, we were able to run a mock server to build and test the UI before integrating with the backend, as well as verify the backend implementation against the specifications. We use Prism to run the mock server and Dredd to verify the server implementation (although I prefer to use Schemathesis now). This allows projects to develop more clearly and efficiently .
Fix API release process
Even if there is an API document, if it is not tested against the API specification before release, the role of the document will become limited. While the documentation itself helps us understand how the API works, its real power lies in its function as a verification tool—verifying that the server is implemented correctly.
To ensure that the API server is running as expected, I incorporated the Dredd test suite into the continuous integration server. No one can merge and publish new code unless it passes Dredd validation and conforms to the API specification. This step allows the team to avoid inadvertent modifications to the API server. From now on, all changes to the server need to be documented in advance, and the API server must be ensured to follow these changes before merging or publishing. This approach ensures the stability and consistency of the server.
Choose the right API development framework
Previously, the team implemented the API server using the Flask framework, a popular Python framework for building web applications. They built the API with basic Flask and wrote a lot of custom code to validate the API payload. This is a common mistake made by many inexperienced API developers.
Building a custom API validation layer is not undesirable, but overreliance on this approach can lead to reinventing the wheel. The API requires complex validation logic to handle the payload and URL (path and query parameters), which must be implemented throughout the API. Therefore, if you insist on building your own API verification layer, you will eventually produce an API framework. However, there are many excellent API development frameworks available, so why not choose one of them?
For Flask specifically, there are several options to consider. But not all frameworks are created equal. For example, flasgger, restx (successor of flask-restplus), flask-RESTful and flask-smorest, etc. With so many choices, how do you make a decision?
When choosing a REST API development framework, the following factors should be considered:
· OpenAPI support : The framework should allow easy building of REST APIs and automatically generate API documentation, ensuring correctness of payload verification. Reference frameworks include Flasgger and flask-smorest.
· Robust data validation : Validating payloads requires a robust data validation library to handle various attributes and types. The library should support optional and required properties such as string formats (ISO date and UUID), as well as strict and relaxed type validation. In the Python ecosystem, pydantic and marshmallow are excellent data validation libraries, and Flasgger and flask-smorest can use marshmallow.
Comprehensive validation : The framework should support validation of request payloads, response payloads, URL path parameters, and URL query parameters. Some libraries may only validate the request payload and ignore URL parameters or response payload. Make sure the framework provides at least one way to enforce validation rules. If using marshmallow, the response payload can be verified directly using its model.
· Maturity : Choose a library that is mature, stable, and has active community support. It should have good documentation and the ability to quickly resolve user issues.
After evaluating the above factors, we selected flask-smorest, a Flask plug-in that supports the use of marshmallow to easily build data verification for REST APIs. It can simplify the data verification process and reduce the amount of custom code. Both request and response payloads are now properly validated, and marshmallow handles validation of URL queries and path parameters as well.
Choosing the right API framework allows us to have a functioning API. Because the framework handles the details of the API layer, we are able to focus more on the business logic and data model of the application, resulting in increased development speed and the ability to release better software more frequently.
Consider everything to build a great API
In today's Internet, APIs are everywhere and inevitable. However, truly building a good API is a complex task. Just like writing code that is readable and easy to maintain, building a high-quality, easy-to-use and easy-to-modify interface is also a challenge in API development.
Providing a high-quality API is difficult because it needs to be consistent with business requirements, that is, the client and server must use the same conventions and specifications, otherwise integration will be difficult to achieve.
Achieving this goal requires considering multiple aspects: gathering customer requirements, translating the requirements into technical details, designing the API, writing documentation, choosing an appropriate API framework and using it correctly, testing the API, and ensuring that the implementation meets the design specifications. This does not include aspects such as API security, deployment and operation.
When it comes to building business-critical APIs, the advice is not to leave it to junior developers alone. They can participate, but a senior developer is needed to guide their work. If you have no API development experience, follow the best practice path: design the API first, then write documentation, implement it according to the specification, and verify it. Don’t reinvent the wheel, choose the right framework!
Although it may take some time to learn OpenAPI, understand the API testing framework, research the API development framework, etc., it is worth it. If not, you will get stuck in a vicious cycle of API integration and software quality issues that will hinder project progress and incur huge costs to fix problems.
If you have sufficient resources and experienced API developers, the probability of building the API correctly will be greatly increased, and you can save a lot of money. In addition, we must also pay attention to the management of APIs. Many problems lie not only in the early development and expenditure costs, but also in hidden code problems during the development process and unclear handovers of responsible employees, which may cause hidden dangers. Therefore, Power Simple Integration believes that API management can use third-party tools to manage APIs in one stop, with multiple roles involved, to avoid shirk of responsibility and improve development efficiency.