fast_csv_loader¶
fast_csv_loader is a fast and memory efficient way to load large CSV files (Timeseries data) into Pandas Dataframes.
Python version: >= 3.8
GitHub Source: BennyThadikaran/fast_csv_loader
The csv_loader function efficiently loads a partial portion of a large CSV file containing time-series data into a pandas DataFrame.
The function allows:
Loading the last N lines of the file.
Loading the last N lines from a specific date.
It can load any type of time-series (both timezone aware and Naive). It handles both daily or intraday data.
It is useful for loading large datasets that may not fit entirely into memory.
It also improves program execution time, when iterating or loading a large number of CSV files.
Installation¶
To use fast_csv_loader, first install it using pip:
pip install fast-csv-loader