python - How to perform data analysis on 20 million pieces of structured data

Question

Prepare for time series analysis, groupby analysis and trend prediction. Is using Python dataframe enough?

阿神 · Answer

1. If it is a sql server database, there is a built-in data warehouse that can do some olap.
2. If you have a hadoop cluster, you can also import it to the cluster for analysis operations. 3. If none of the above are available, and your The machine configuration is good and the memory is sufficient, so just use pandas to do it