Section I: The purpose of this book
About the author
Zhang Xiaoting (Mike Cheers), 1982 was born, the vast prairie of Inner Mongolia is my hometown.
Not highly educated, not talking about the talk, have a technology that is the persistent pursuit of self-worth.
Who am I, really does not matter, I'm a senior developer, I'm an architect, I was technical manager, these are my, not a cent relationship with you. The most important is to give the reader what I bring. The next day, take a look at the book to give you bring any surprises, perhaps in the end, you will not remember my name, it does not matter, believe me, it does not matter.
About the Book
This book is part of the "broken border of the Road" series of books of technology, the system will conduct a comprehensive explanation for the use of C # to develop data acquisition (commonly known as crawlers), writing intention is to hope that through this book so that readers can grasp the depth of about reptiles aspects. The so-called "broken border" from the practitioner breakthrough in the realm of practice on the road. There is also the author of his mouth a deep earnest expectation of the audience, hoping to help the reader in on the technology practice of the road, to "martial arts pinnacle."
This series of books known as "reading to lose." When knowledge and technology becoming FMCG, although I also do not like it, but looking back, had entered the ranks of practice, the use of ASP, VBScript, etc., have long been out of sight, I believe that it is running out . Era in the development, technological advances, most practitioners are also accustomed to daily or even new cutting-edge technology to follow the trend of the day and night, though already tired, but never bored. This is also the series of books expectations on the name of the practitioners, hoping to help readers detailed combing knowledge, the formation of knowledge in the mind, and through a variety of angles analysis deeper impression, keep in mind, the knowledge move into my mind from the boring text . Well, then, this book will be lost! Do not stop, there is more knowledge awaits us.
This series of books, will try to use the most current version of popular even cutting-edge technology, but, after all, writing also takes a lot of time and energy, may you see the book, it has lost its dazzling brilliance, but still want to help those product lines not updated technical framework needs or plans. Similarly, if the reader some knowledge there is doubt, may also be a leak filled through the book, I believe will quickly master the new technology is also helpful.
The purpose of writing this book is very clear, delve into the technical details of C # and reptiles in the practical work, we need to pay attention to the "pit."
This book is not introductory books, and do not have the knowledge level division, because the actual production process, the requirement is high, there is no product manager ran will say to you: "Brother, this week let's first take a first entry-level mall with the next week and then upgrade to a mid-level, and then at the end we upgrade to advanced, you say, OK?. " I believe he said so, you are very satisfied, because you see so many introductory books, very used, very adapt to even very fond of, but not to the end, the product manager may have already departed.
Section II: Overview of reptiles developed using C #
Main features
- Distributed: reptile from a single point to the cluster development has been a big trend, of course, as the case may be, not all the needs of the cluster must be made, but I hope that employees have the ability to expand it.
- Two sets of templates: Templates collection and analysis templates, when our acquisition target is not fixed, we hope in time to switch to the next target, it is possible to write less code as possible through the configuration.
- Multithreading: data acquisition, data processing, multithreading is essential basic skills.
- Application cache: cache will help us improve the overall performance of the system, but it is also a double-edged sword, in practical applications, require multi considerations, be used with caution.
- Data Storage: There are a lot of data storage media, ideas, methods, nothing is best, just like the knife or the sword, their own strengths, recommended during the development phase, to use a handy, it would not promote early adopters, but production environment, or the use of the most suitable.
- Data source: in the acquisition process, we often encounter a number of obstacles, from technical problems itself, personal ability or the data source itself, then, we need to broaden thinking, do not hang in a tree, try a change in thinking and path , may be able to achieve better results.
- Scalable: the book as much as possible so that the reader can develop a scalable system of reptiles, including but not limited to: nodes can be expanded, flexible configuration, custom interfaces.
These features will be explained in detail into the specific sections.
-
Section III: development environment ready
MVC wants to start our trip, the development environment is essential, but the book does not want to waste too much ink on the basis of environment building, just a list of basic environmental needs and IDE development environment used in this book.
- Windows operating system
- Visual Studio 2017
- Database (in order to avoid disagreement on what database you use this book to make use of the file system, more neutral, as to what kind of production database environments, depends on many factors, the following sections will give some suggestions)
- But readers still need to have some knowledge of C # and database side, the stronger the better! Other books in this series can be found in a series of books.