Usage

Example

from datetime import datetime
from pathlib import Path

import pandas as pd

from fast_csv_loader import csv_loader

file = Path("path/to/your/timeseries_data.csv")

# Load 160 lines of data from end of file
df = csv_loader(file_path=file)

print(df)

# Load last 500 lines of data upto 10th May 2024
df = csv_loader(
    file_path=file,
    period=500,
    end_date=datetime(2024, 5, 10),
)

# If `Date` is not default datetime column, specify it using `date_column`
df = csv_loader(file_path=file, date_column="time")

# If pandas is unable to parse the date column, specify the format using `date_format`
df = csv_loader(file_path=file, date_format="%d/%m/%Y")

# Increase the `chunk_size` to optimize performance based of your specific needs.
df = csv_loader(file_path=file, chunk_size=1024 * 10)

API

csv_loader.csv_loader(file_path: Path, period: int = 160, end_date: datetime | None = None, date_column: str = 'Date', date_format: str | None = None, chunk_size: int = 6144) → DataFrame

Load a CSV file with timeseries data in chunks from the end.

Could return an empty DataFrame, if no data was found. Use df.empty to check if the DataFrame is empty before further processing.

Parameters:

  • file_path (pathlib.Path) – The path to the CSV file to be loaded.

  • period (int) – Number of lines/candles to return. The default is 160.

  • end_date (Optional[datetime]) – Load N lines up to this date. If None, will load the last N lines from the file. If the date is provided, load the last N lines from this date.

  • date_column (str) – Name of the date column. Defaults to Date.

  • date_format (Optional[str]) – Custom date format in case pandas is unable to parse the date column.

  • chunk_size (int) – The size of data chunks loaded into memory. The default is 6144 bytes (6 KB).

Returns:

A DataFrame containing the loaded timeseries data.

Return type:

pd.DataFrame

Raises:

IndexError – if end_date is provided but not within the boundary of the data.