[Vnpy quantitative investment from scratch] 7. Access to third-party data sources and download data regularly
overview
In the last lesson, we explained how to switch the database of vnpy to mysql to facilitate the isolation of data from software. In this lesson, we will take Tianqin tqsdk as an example to introduce how to store the data of external service providers into mysql and finally use it for vnpy.
Why do you need to access third-party data
Different data service providers may have great differences in data quality, timeliness, cost, and interface ease of use. We need to deeply understand the differences between various data APIs, as well as the needs of our own strategies, and finally choose the data source that suits us. .
Taking the author as an example, since the strategy I use strongly relies on the opening price of the morning market every day, I need to be able to obtain the accurate opening price at any time. Here I have to talk about the k-line synthesis mechanism of vnpy. vnpy only queries the existing data through datafeed or database during load(N), and then all k-lines generated during the real trading operation are synthesized from the tick data pushed by the exchange market server. This will cause a problem. If the k-line data as of the current moment is not stored in the database, it is necessary to start vnpy before 9:00 and receive the first tick from the exchange to obtain the opening price of the day. Once the vnpy service is restarted due to various reasons, the data that has been acquired that day will be lost at this time, unless all the data that has been generated that day has been stored in the database before starting.
In addition, if you want to ensure that the startup is completed before 9 o'clock and the first tick is received, sometimes there will be problems. Both the init and start methods of vnpy are processed asynchronously through multiple threads, and the completion time depends on the running speed of the strategy itself, so in order to ensure that when starting the start The strategy has been loaded, and the waiting time of the main thread in the middle is generally configured for a long time. The author has encountered the situation of not subscribing to the contract data many times after the startup, that is, the data was not received after the market opened. At this time, the service can only be restored by restarting immediately. At this time, no matter whether it is restarted or not, the first tick has been missed. data. The tick data itself is not the real full amount of data, it is only a transaction slice at a certain moment in all moments of the exchange, which is why we often find that the ticks we accept are synthesized