ManagedkdbInsights/basic_tick_V3/manual_eod.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2ec5d9d9-becb-4156-97fb-62354335c047",
   "metadata": {},
   "source": [
    "# Manual Running End of Day (EOD)\n",
    "\n",
    "This notebook demonstrates (using kdb q magic cells) how to connect to and manually execute the end of day (EOD) process on the RDB.\n",
    "\n",
    "## Reference\n",
    "[PyKX Jupyter q Magic Command](https://code.kx.com/pykx/2.5/getting-started/q_magic_command.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d628c863-25f3-4ad3-a66c-fd1a92910dc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.simplefilter(action='ignore', category=FutureWarning)\n",
    "\n",
    "import os\n",
    "import boto3\n",
    "import json\n",
    "import datetime\n",
    "\n",
    "import pykx as kx\n",
    "\n",
    "from env import *\n",
    "from managed_kx import *\n",
    "\n",
    "# Cluster names and database\n",
    "from config import *\n",
    "\n",
    "# Specific cluster to connect to\n",
    "CLUSTER_NAME = RDB_CLUSTER_NAME\n",
    "\n",
    "# set pykx local q console width and height\n",
    "kx.q.system.display_size = [50, 1000]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6925c3e1-311b-4875-85f7-d5b13f60c295",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using credentials and create service client\n",
    "session = boto3.Session()\n",
    "\n",
    "# create finspace client\n",
    "client = session.client(service_name='finspace')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a19be40b-d706-44d8-bde2-65112174c5f8",
   "metadata": {},
   "source": [
    "# Execute End of Day (EOD) Function on RDB\n",
    "Use components of the connection string and the pykx included q magic to connect to a cluster and interact with it, useful for remote debugging.\n",
    "\n",
    "- Get the connection string   \n",
    "  - must have permission to environment and cluster to get a connection string to the cluster\n",
    "- Check state of the dataview _before_ running EOD\n",
    "- Call the eod of day function on the RDB (.rdb.eod) with today's date\n",
    "- Check state of the dataview _after_ EOD completes\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdac885f-cce8-474b-b3db-b91d6470b090",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the full connection string\n",
    "conn_str = get_kx_connection_string(client, \n",
    "                                    environmentId=ENV_ID, clusterName=CLUSTER_NAME, \n",
    "                                    userName=KDB_USERNAME, boto_session=session)\n",
    "\n",
    "# parse the connection string into its components\n",
    "host, port, username, password = parse_connection_string(conn_str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b85743f-35a3-4a8c-981f-93fe518d503d",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%q --host $host --port $port --user $username --pass $password\n",
    "\n",
    "/ current state of the dataview\n",
    "dview:.aws.get_kx_dataview[.rdb.database;.rdb.dbView];\n",
    "dview\n",
    "\"\"\n",
    "/ dataview changeset details\n",
    ".aws.get_changeset[.rdb.database;dview`changeset_id]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "106dac99-3821-4e28-b0c7-178d46fc285d",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%q --host $host --port $port --user $username --pass $password\n",
    "\n",
    "/ Run EOD for today\n",
    ".u.end[.z.d]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f69f6da5-ace0-4fb2-9207-1e4fbaa8bd8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%q --host $host --port $port --user $username --pass $password\n",
    "\n",
    "/ new state of dataview\n",
    "dview:.aws.get_kx_dataview[.rdb.database;.rdb.dbView];\n",
    "dview\n",
    "\"\"\n",
    "/ dataview changeset details\n",
    ".aws.get_changeset[.rdb.database;dview`changeset_id]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3cf3c844-d7a3-4b27-a11d-6ad00574ea91",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print( f\"Last Run: {datetime.datetime.now()}\" )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61714b7f-b95f-48c2-b87a-d25178906db2",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p310",
   "language": "python",
   "name": "conda_pytorch_p310"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}