macrosynergy.download.dataquery#

Interface for downloading data from the JPMorgan DataQuery API. This module is not intended to be used directly, but rather through macrosynergy.download.jpmaqs.py. However, for a use cases independent of JPMaQS, this module can be used directly to download data from the JPMorgan DataQuery API.

validate_response(response, user_id)[source]#

Validates a response from the API. Raises an exception if the response is invalid (e.g. if the response is not a 200 status code).

Parameters:

response (requests.Response) – response object from requests.request().

Raises:
Returns:

response as a dictionary. If the response is not valid, this function will raise an exception.

Return type:

dict

format_invalid_response_msg(response, user_id)[source]#

This function formats an error message for an invalid response from the API. Should only be called if there is an error in the response (as the functions adds the response text to the error message).

Return type:

str

request_wrapper(url, headers=None, params=None, method='get', tracking_id=None, proxy=None, cert=None, **kwargs)[source]#

Wrapper for requests.request() that handles retries and logging. All parameters and kwargs are passed to requests.request().

Parameters:

  • url (str) – URL to request.

  • headers (dict) – headers to pass to requests.request().

  • params (dict) – params to pass to requests.request().

  • method (str) – HTTP method to use. Must be one of “get” or “post”. Defaults to “get”.

  • kwargs (dict) – kwargs to pass to requests.request().

  • tracking_id (str) – default None, unique tracking ID of request.

  • proxy (dict) – default None, dictionary of proxy settings for request.

  • cert (Tuple[str, str]) – default None, tuple of string for filename of certificate and key.

Raises:
Returns:

response as a dictionary.

Return type:

dict

class DataQueryOAuth(client_id, client_secret, proxy=None, token_url='https://authe.jpmchase.com/as/token.oauth2', dq_base_url='https://api-developer.jpmorgan.com/research/dataquery-authe/api/v2', dq_resource_id='JPMC:URI:RS-06785-DataQueryExternalApi-PROD', application_name='DataQueryHttpAPI', **kwargs)[source]#

Bases: JPMorganOAuth

class DataQueryCertAuth(username, password, crt, key, proxy=None)[source]#

Bases: object

Class for handling certificate based authentication for the DataQuery API.

Parameters:

  • username (str) – username for the DataQuery API.

  • password (str) – password for the DataQuery API.

  • crt (str) – path to the certificate file.

  • key (str) – path to the key file.

Raises:

  • AssertionError – if any of the parameters are of the wrong type.

  • FileNotFoundError – if certificate or key file is missing from filesystem.

  • Exception – other exceptions may be raised by underlying functions.

get_headers()[source]#
Return type:

dict

get_auth()[source]#

Returns a dictionary with the authentication information, in the same format as the macrosynergy.download.dataquery.OAuth.get_auth() method.

Return type:

Dict[str, Union[str, Tuple[str, str], None]]

validate_download_args(expressions, start_date, end_date, show_progress, endpoint, calender, frequency, conversion, nan_treatment, reference_data, retry_counter, delay_param, batch_size)[source]#

Validate the arguments passed to the download_data() method.

Raises:

  • TypeError – if any of the arguments are of the wrong type.

  • ValueError – if any of the arguments are semantically incorrect.

Returns:

True if all arguments are valid.

Return type:

bool

class DataQueryInterface(client_id=None, client_secret=None, crt=None, key=None, username=None, password=None, proxy=None, oauth=True, debug=False, batch_size=20, check_connection=True, base_url='https://api-developer.jpmorgan.com/research/dataquery-authe/api/v2', token_url='https://authe.jpmchase.com/as/token.oauth2', suppress_warning=True, custom_auth=None, verify=True)[source]#

Bases: object

High level interface for the DataQuery API. When using OAuth authentication:

Parameters:

  • client_id (str) – client ID for the OAuth application.

  • client_secret (str) – client secret for the OAuth application. When using certificate authentication:

  • crt (str) – path to the certificate file.

  • key (str) – path to the key file.

  • username (str) – username for the DataQuery API.

  • password (str) – password for the DataQuery API.

  • oauth (bool) – whether to use OAuth authentication. Defaults to True.

  • debug (bool) – whether to print debug messages. Defaults to False.

  • concurrent (bool) – whether to use concurrent requests. Defaults to True.

  • batch_size (int) – default 20, number of expressions to send in a single request. Must be a number between 1 and 20 (both included).

  • check_connection (bool) – whether to send a check_connection request. Defaults to True.

  • base_url (str) – base URL for the DataQuery API. Defaults to OAUTH_BASE_URL if oauth is True, CERT_BASE_URL otherwise.

  • token_url (str) – token URL for the DataQuery API. Defaults to OAUTH_TOKEN_URL.

  • suppress_warnings (bool) – whether to suppress warnings. Defaults to True.

  • custom_auth (Any) – custom authentication object. When specified oauth must be False and the object must have a get_auth method. Defaults to None.

Raises:

  • TypeError – if any of the parameters are of the wrong type.

  • ValueError – if any of the parameters are semantically incorrect.

  • InvalidResponseError – if the response from the server is not valid.

  • DownloadError – if the download fails to complete after a number of retries.

  • HeartbeatError – if the heartbeat (check connection) fails.

  • Exception – other exceptions may be raised by underlying functions.

check_connection(verbose=False, raise_error=False)[source]#

Check the connection to the DataQuery API using the Heartbeat endpoint.

Parameters:

verbose (bool) – whether to print a message if the heartbeat is successful. Useful for debugging. Defaults to False.

Raises:

HeartbeatError – if the heartbeat fails.

Returns:

True if the connection is successful, False otherwise.

Return type:

bool

get_catalogue(group_id='JPMAQS', page_size=1000, verbose=True)[source]#

Method to get the JPMaQS catalogue. Queries the DataQuery API’s Groups/Search endpoint to get the list of tickers in the JPMaQS group. The group ID can be changed to fetch a different group’s catalogue.

Parameters:

  • group_id (str) – the group ID to fetch the catalogue for. Defaults to “JPMAQS”.

  • page_size (int) – the number of tickers to fetch in a single request. Defaults to 1000 (maximum allowed by the API).

Raises:

ValueError – if the response from the server is not valid.

Returns:

list of all tickers in the requested group.

Return type:

List[str]

download_data(expressions, start_date='2000-01-01', end_date=None, show_progress=False, endpoint='/expressions/time-series', calender='CAL_ALLDAYS', frequency='FREQ_DAY', conversion='CONV_LASTBUS_ABS', nan_treatment='NA_NOTHING', reference_data='NO_REFERENCE_DATA', retry_counter=0, delay_param=0.25, batch_size=None, *args, **kwargs)[source]#

Download data from the DataQuery API.

Parameters:

  • expressions (List[str]) – list of expressions to download.

  • start_date (str) – start date for the data in the ISO-8601 format (YYYY-MM-DD).

  • end_date (str) – end date for the data in the ISO-8601 format (YYYY-MM-DD).

  • show_progress (bool) – whether to show a progress bar for the download.

  • endpoint (str) – endpoint to use for the download.

  • calender (str) – calendar setting to use for the download.

  • frequency (str) – frequency of data points to use for the download.

  • conversion (str) – conversion setting to use for the download.

  • nan_treatment (str) – NaN treatment setting to use for the download.

  • reference_data (str) – reference data to pass to the API kwargs.

  • retry_counter (int) – number of times the download has been retried.

  • delay_param (float) – delay between requests to the API.

Raises:
Returns:

list of dictionaries containing the response data.

Return type:

List[Dict]