macrosynergy.management.types.qdf

class QuantamentalDataFrame(df=None, categorical=True, _initialized_as_categorical=None)

Bases: QuantamentalDataFrameBase

Type extension of pd.DataFrame for Quantamental DataFrames. Usage: >>> df: pd.DataFrame = load_data() >>> qdf = QuantamentalDataFrame(df)

is_categorical()

Returns True if the QuantamentalDataFrame is categorical.

Returns:

True if the QuantamentalDataFrame is categorical

Return type:

bool

to_categorical()

Converts the QuantamentalDataFrame to a categorical DataFrame.

Returns:

The QuantamentalDataFrame with categorical columns.

Return type:

QuantamentalDataFrame

to_string_type()

Converts the QuantamentalDataFrame to a string DataFrame.

Returns:

The QuantamentalDataFrame with string columns.

Return type:

QuantamentalDataFrame

to_original_dtypes()

Converts the QuantamentalDataFrame to its original dtypes (using the InitialisedAsCategorical attribute).

Returns:

The QuantamentalDataFrame with its original dtypes. The dtype is determined by the InitialisedAsCategorical attribute. The output dtype will be either ‘category’ or ‘object’.

Return type:

QuantamentalDataFrame

list_tickers()

List all tickers in the QuantamentalDataFrame.

Returns:

A list of all tickers in the QuantamentalDataFrame.

Return type:

List[str]

add_ticker_column()

Add a ticker column to the QuantamentalDataFrame. ticker is a combination of cid and xcat columns. i.e. ticker = cid_xcat.

Returns:

The QuantamentalDataFrame with a ticker column.

Return type:

QuantamentalDataFrame

drop_ticker_column()

Drop the ticker column from the QuantamentalDataFrame.

Raises:

ValueError – If no ticker column is found in the DataFrame.

Returns:

The QuantamentalDataFrame without the ticker column.

Return type:

QuantamentalDataFrame

reduce_df(cids=None, xcats=None, start=None, end=None, blacklist=None, out_all=False, intersect=False)

Filter DataFrame by cids, xcats, and start & end dates.

Parameters:

• cids (Optional[Sequence[str]], optional) – List of CIDs to filter by, by default None

• xcats (Optional[Sequence[str]], optional) – List of XCATs to filter by, by default None

• start (Optional[str], optional) – Start date to filter by, by default None

• end (Optional[str], optional) – End date to filter by, by default None

• blacklist (Dict[str, Sequence[Union[str, pd.Timestamp]]], optional) – Blacklist to apply to the DataFrame, by default None

• out_all (bool, optional) – If True, return the filtered DataFrame, the filtered XCATs, and the filtered CIDs, by default False

Return type:

Union[QuantamentalDataFrame, Tuple[QuantamentalDataFrame, List[str], List[str]]]

Returns:

• QuantamentalDataFrame – The filtered QuantamentalDataFrame. (if out_all=False, default)

• Tuple[QuantamentalDataFrame, List[str], List[str]] – The filtered QuantamentalDataFrame, the filtered XCATs, and the filtered CIDs. (if out_all=True)

reduce_df_by_ticker(tickers, start=None, end=None, blacklist=None)

Filter DataFrame by ticker, start & end dates.

Parameters:

• tickers (Sequence[str]) – List of tickers to filter by

• start (Optional[str], optional) – Start date to filter by, by default None

• end (Optional[str], optional) – End date to filter by, by default None

• blacklist (Mapping[str, Sequence[Union[str, pd.Timestamp]]], optional) – Blacklist to apply to the DataFrame, by default None

Returns:

The filtered QuantamentalDataFrame.

Return type:

QuantamentalDataFrame

apply_blacklist(blacklist)

Apply a blacklist to the QuantamentalDataFrame.

update_df(df_add, xcat_replace=False)

Update the QuantamentalDataFrame with a new DataFrame.

Parameters:

• df_add (pd.DataFrame) – DataFrame to update the QuantamentalDataFrame with

• xcat_replace (bool, optional) – If True, replace the XCATs in the QuantamentalDataFrame with the XCATs in df_add, by default False

Returns:

The updated QuantamentalDataFrame.

Return type:

QuantamentalDataFrame

add_nan_series(ticker, start=None, end=None)

Add a NaN series to the QuantamentalDataFrame.

Parameters:

• ticker (str) – Ticker to add the NaN series to

• start (Optional[str], optional) – Start date of the NaN series, by default None

• end (Optional[str], optional) – End date of the NaN series, by default None

Returns:

The QuantamentalDataFrame with the NaN series added.

Return type:

QuantamentalDataFrame

drop_nan_series(column='value', raise_warning=True)

Drop NaN series from the QuantamentalDataFrame.

Parameters:

• column (str, optional) – Column to check for NaN series, by default “value”

• raise_warning (bool, optional) – If True, raise a warning if NaN series are dropped, by default True

Returns:

The QuantamentalDataFrame with NaN series dropped

Return type:

QuantamentalDataFrame

rename_xcats(xcat_map=None, select_xcats=None, postfix=None, prefix=None, name_all=None, fmt_string=None)

Rename xcats in the QuantamentalDataFrame.

Parameters:

• xcat_map (Optional[Mapping[str, str]], optional) – Mapping of xcats to rename, by default None

• select_xcats (Optional[List[str]], optional) – List of xcats to rename, by default None

• postfix (Optional[str], optional) – Postfix to add to the xcats, by default None

• prefix (Optional[str], optional) – Prefix to add to the xcats, by default None

• name_all (Optional[str], optional) – Name to rename all xcats to, by default None

• fmt_string (Optional[str], optional) – Format string to rename xcats, by default None

Returns:

The QuantamentalDataFrame with the xcats renamed.

Return type:

QuantamentalDataFrame

to_wide(value_column='value')

Pivot the QuantamentalDataFrame.

Parameters:

value_column (str, optional) – Column to pivot, by default “value”

Returns:

The pivoted QuantamentalDataFrame, with each ticker as a column with the values of the value_column and the index as the real_date.

Return type:

QuantamentalDataFrame

classmethod from_timeseries(timeseries, ticker, metric='value')

Convert a timeseries DataFrame to a QuantamentalDataFrame.

Parameters:

• timeseries (pd.Series) – Timeseries to convert to a QuantamentalDataFrame

• ticker (str) – Ticker to assign to the timeseries

• metric (str, optional) – Metric to assign to the timeseries, by default “value”

Returns:

The QuantamentalDataFrame created from the timeseries.

Return type:

QuantamentalDataFrame

classmethod from_long_df(df, real_date_column='real_date', value_column='value', cid=None, xcat=None, ticker=None, categorical=True)

Convert a long DataFrame to a QuantamentalDataFrame. This is useful when the DataFrame may contain only a cid or xcat column, or in cases where the cid and xcat columns are not named as such.

Parameters:

• df (pd.DataFrame) – Long DataFrame to convert to a QuantamentalDataFrame

• real_date_column (str, optional) – Column name of the real date, by default “real_date”

• value_column (str, optional) – Column name of the value, by default “value”

• cid (Optional[str], optional) – cid to assign to the DataFrame, by default None. If not specified, the cid column must be present in the DataFrame.

• xcat (Optional[str], optional) – xcat to assign to the DataFrame, by default None

• ticker (Optional[str], optional) – Ticker to assign to the DataFrame, by default None

• categorical (bool, optional) – If True, convert the DataFrame to categorical, by default True

Raises:

• ValueError – If the real_date_column or value_column are not found in the DataFrame, or if ticker is specified with cid or xcat.

• ValueError – If the input DataFrame is empty.

• ValueError – If the cid or xcat columns are not found in the DataFrame, and have not been specified in the function call.

Returns:

The QuantamentalDataFrame created from the long DataFrame.

Return type:

QuantamentalDataFrame

classmethod from_qdf_list(qdf_list, categorical=True)

Concatenate a list of QuantamentalDataFrames into a single QuantamentalDataFrame.

Parameters:

• qdf_list (List[QuantamentalDataFrame]) – List of QuantamentalDataFrames to concatenate

• categorical (bool, optional) – If True, convert the DataFrame to categorical, by default True

Raises:

• TypeError – If any element in the list is not a QuantamentalDataFrame.

• ValueError – If the input list is empty.

Returns:

The concatenated QuantamentalDataFrame.

Return type:

QuantamentalDataFrame

classmethod from_wide(df, value_column='value', categorical=True)

Convert a wide DataFrame to a QuantamentalDataFrame.

Parameters:

• df (pd.DataFrame) – Wide DataFrame to convert to a QuantamentalDataFrame

• value_column (str, optional) – Column to pivot, by default “value”

• categorical (bool, optional) – If True, convert the DataFrame to categorical, by default True

Raises:

• TypeError – If df is not a pandas DataFrame.

• ValueError – If df does not have a datetime index.

• ValueError – If all columns are not in the format ‘cid_xcat’.

Returns:

The QuantamentalDataFrame created from the wide DataFrame.

Return type:

QuantamentalDataFrame

classmethod create_empty_df(cid=None, xcat=None, ticker=None, metrics=['value'], date_range=None, start=None, end=None, categorical=True)

Create an empty QuantamentalDataFrame.

Parameters:

• cid (Optional[str], optional) – cid to assign to the DataFrame, by default None

• xcat (Optional[str], optional) – xcat to assign to the DataFrame, by default None

• ticker (Optional[str], optional) – Ticker to assign to the DataFrame, by default None. If specified, cid and xcat must not be specified.

• metrics (List[str], optional) – Metrics to assign to the DataFrame, by default [“value”]

• date_range (Optional[pd.DatetimeIndex], optional) – Date range to assign to the DataFrame, by default None. If not specified, start and end must be specified.

• start (Optional[str], optional) – Start date to assign to the DataFrame, by default None

• end (Optional[str], optional) – End date to assign to the DataFrame, by default None

• categorical (bool, optional) – If True, convert the DataFrame to categorical, by default True

Returns:

The empty QuantamentalDataFrame.

Return type:

QuantamentalDataFrame

class QuantamentalDataFrameBase(data=None, index=None, columns=None, dtype=None, copy=None)

Bases: DataFrame

Base class to extend pd.DataFrame for Quantamental DataFrames. This class is a parent class to macrosynergy.types.QuantamentalDataFrame.

IndexCols: List[str] = ['real_date', 'cid', 'xcat']

IndexColsSortOrder = ['cid', 'xcat', 'real_date']

Submodules

• macrosynergy.management.types.qdf.base
  • QuantamentalDataFrameMeta
    • QuantamentalDataFrameMeta.IndexCols
  • QuantamentalDataFrameBase
    • QuantamentalDataFrameBase.IndexCols
    • QuantamentalDataFrameBase.IndexColsSortOrder
• macrosynergy.management.types.qdf.classes
  • QuantamentalDataFrame
    • QuantamentalDataFrame.is_categorical()
    • QuantamentalDataFrame.to_categorical()
    • QuantamentalDataFrame.to_string_type()
    • QuantamentalDataFrame.to_original_dtypes()
    • QuantamentalDataFrame.list_tickers()
    • QuantamentalDataFrame.add_ticker_column()
    • QuantamentalDataFrame.drop_ticker_column()
    • QuantamentalDataFrame.reduce_df()
    • QuantamentalDataFrame.reduce_df_by_ticker()
    • QuantamentalDataFrame.apply_blacklist()
    • QuantamentalDataFrame.update_df()
    • QuantamentalDataFrame.add_nan_series()
    • QuantamentalDataFrame.drop_nan_series()
    • QuantamentalDataFrame.rename_xcats()
    • QuantamentalDataFrame.to_wide()
    • QuantamentalDataFrame.from_timeseries()
    • QuantamentalDataFrame.from_long_df()
    • QuantamentalDataFrame.from_qdf_list()
    • QuantamentalDataFrame.from_wide()
    • QuantamentalDataFrame.create_empty_df()
• macrosynergy.management.types.qdf.methods
  • get_col_sort_order()
  • change_column_format()
  • qdf_to_categorical()
  • qdf_to_string_index()
  • check_is_categorical()
  • apply_blacklist()
  • reduce_df()
  • reduce_df_by_ticker()
  • update_df()
  • update_tickers()
  • update_categories()
  • qdf_to_wide_df()
  • add_ticker_column()
  • rename_xcats()
  • create_empty_categorical_qdf()
  • add_nan_series()
  • drop_nan_series()
  • qdf_from_timeseries()
  • concat_qdfs()