courses/machine_learning/packaged/sdsoln.ipynb

{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "<h1> Structured Data Solution </h1>\n", "\n", "In this notebook, we will use the structured data package in Datalab to build a model to predict taxifares." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import os\n", "PROJECT = 'cloud-training-demos' # CHANGE THIS\n", "BUCKET = 'cloud-training-demos-ml' # CHANGE THIS\n", "REGION = 'us-central1' # CHANGE THIS\n", "\n", "os.environ['PROJECT'] = PROJECT # for bash\n", "os.environ['BUCKET'] = BUCKET # for bash\n", "os.environ['REGION'] = REGION # for bash" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%bash\n", "echo \"project=$PROJECT\"\n", "echo \"bucket=$BUCKET\"\n", "echo \"region=$REGION\"\n", "gcloud config set project $PROJECT\n", "gcloud config set compute/region $REGION\n", "gcloud beta ml init-project -q" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "No handlers could be found for logger \"oauth2client.contrib.multistore_file\"\n", "WARNING:root:Couldn't find python-snappy so the implementation of _TFRecordUtil._masked_crc32c is not as fast as it could be.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "tf 1.0.0\n", "sd 0.0.1\n", "cml 0.1.9.1-alpha\n" ] } ], "source": [ "import tensorflow as tf\n", "import datalab.ml as ml\n", "import google.cloud.ml as cml\n", "import datalab_solutions.structured_data as sd\n", "from tensorflow.python.lib.io import file_io\n", "import json\n", "import shutil\n", "\n", "print('tf ' + str(tf.__version__))\n", "print('sd ' + str(sd.__version__))\n", "print('cml ' + str(cml.__version__))" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "INDIR = '../feateng/sample'\n", "OUTDIR = '.'\n", "\n", "# for bash\n", "os.environ['OUTDIR'] = OUTDIR" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<h2> Set up schema file </h2>\n", "\n", "Schema of training/test. Same format as BigQuery. STRING/INTEGER/FLOAT only." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting ./taxifare.json\n" ] } ], "source": [ "%writefile $OUTDIR/taxifare.json\n", "[\n", " {\n", " \"mode\": \"NULLABLE\",\n", " \"name\": \"fare_amount\",\n", " \"type\": \"FLOAT\"\n", " }, \n", " {\n", " \"mode\": \"NULLABLE\",\n", " \"name\": \"dayofweek\",\n", " \"type\": \"STRING\"\n", " },\n", " {\n", " \"mode\": \"NULLABLE\",\n", " \"name\": \"hourofday\",\n", " \"type\": \"STRING\"\n", " },\n", " {\n", " \"mode\": \"NULLABLE\",\n", " \"name\": \"pickuplon\",\n", " \"type\": \"FLOAT\"\n", " },\n", " {\n", " \"mode\": \"NULLABLE\",\n", " \"name\": \"pickuplat\",\n", " \"type\": \"FLOAT\"\n", " },\n", " {\n", " \"mode\": \"NULLABLE\",\n", " \"name\": \"dropofflon\",\n", " \"type\": \"FLOAT\"\n", " },\n", " {\n", " \"mode\": \"NULLABLE\",\n", " \"name\": \"dropofflat\",\n", " \"type\": \"FLOAT\"\n", " },\n", " {\n", " \"mode\": \"NULLABLE\",\n", " \"name\": \"passengers\",\n", " \"type\": \"FLOAT\"\n", " },\n", " {\n", " \"mode\": \"REQUIRED\",\n", " \"name\": \"key\",\n", " \"type\": \"STRING\"\n", " } \n", "]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<h2> Preprocessing </h2>\n", "\n", "The first step of preprocessing is to compute the min, max, etc. for scaling purposes." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "!rm -rf $OUTDIR/taxi_preproc $OUTDIR/taxi_trained" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Starting local preprocessing.\n", "Local preprocessing done.\n" ] } ], "source": [ "train_csv = ml.CsvDataSet(\n", " file_pattern=os.path.join(INDIR, 'train*'),\n", " schema_file=os.path.join(OUTDIR, 'taxifare.json'))\n", "sd.local_preprocess(\n", " dataset=train_csv,\n", " output_dir=os.path.join(OUTDIR, 'taxi_preproc'),\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The second step is to specify the feature columns and transformations. The target and key transforms are required. Everything else is optional." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [], "source": [ "transforms = {\n", " \"fare_amount\": {\"transform\": \"target\"},\n", " \"key\": {\"transform\": \"key\"}, \n", " \"dayofweek\": {\"transform\": \"one_hot\"},\n", " \"hourofday\": {\"transform\": \"embedding\", \"embedding_dim\": 2}, # group-combine the hour\n", "}\n", "file_io.write_string_to_file(os.path.join(OUTDIR, 'taxi_preproc/transforms.json'),\n", " json.dumps(transforms, indent=2))" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "numerical_analysis.json\n", "schema.json\n", "transforms.json\n", "vocab_dayofweek.csv\n", "vocab_hourofday.csv\n", "vocab_key.csv\n" ] } ], "source": [ "%bash\n", "ls $OUTDIR/taxi_preproc" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\r\n", " \"passengers\": {\r\n", " \"max\": 6.0,\r\n", " \"mean\": 1.714834159189322,\r\n", " \"min\": 1.0\r\n", " },\r\n", " \"fare_amount\": {\r\n", " \"max\": 130.12,\r\n", " \"mean\": 11.346521137449155,\r\n", " \"min\": 2.5\r\n", " },\r\n", " \"pickuplat\": {\r\n", " \"max\": 41.591743,\r\n", " \"mean\": 40.751719439603896,\r\n", " \"min\": 40.27725\r\n", " },\r\n", " \"dropofflat\": {\r\n", " \"max\": 41.57285,\r\n", " \"mean\": 40.75180645182645,\r\n", " \"min\": 40.303627\r\n", " },\r\n", " \"pickuplon\": {\r\n", " \"max\": -73.137393,\r\n", " \"mean\": -73.97476843613309,\r\n", " \"min\": -75.26911\r\n", " },\r\n", " \"dropofflon\": {\r\n", " \"max\": -73.137393,\r\n", " \"mean\": -73.97400344112786,\r\n", " \"min\": -74.417107\r\n", " }\r\n", "}" ] } ], "source": [ "!cat $OUTDIR/taxi_preproc/num*json" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Wed\r\n", "Sun\r\n", "Fri\r\n", "Tue\r\n", "Mon\r\n", "Thu\r\n", "Sat" ] } ], "source": [ "!cat $OUTDIR/taxi_preproc/vocab_day*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<h2> Local Training and prediction </h2>\n", "\n", "Train using the preproprocessed data." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Starting local training.\n", "WARNING:tensorflow:The default value of combiner will change from \"sum\" to \"sqrtn\" after 2016/11/01.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:The default value of combiner will change from \"sum\" to \"sqrtn\" after 2016/11/01.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:The default value of combiner will change from \"sum\" to \"sqrtn\" after 2016/11/01.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:The default value of combiner will change from \"sum\" to \"sqrtn\" after 2016/11/01.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:The default value of combiner will change from \"mean\" to \"sqrtn\" after 2016/11/01.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:The default value of combiner will change from \"mean\" to \"sqrtn\" after 2016/11/01.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Using config: {'_save_checkpoints_secs': 600, '_num_ps_replicas': 0, '_keep_checkpoint_max': 5, '_tf_random_seed': None, '_task_type': None, '_environment': 'local', '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fd5e98bf3d0>, '_tf_config': gpu_options {\n", " per_process_gpu_memory_fraction: 1.0\n", "}\n", ", '_task_id': 0, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_evaluation_master': '', '_keep_checkpoint_every_n_hours': 10000, '_master': ''}\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Using config: {'_save_checkpoints_secs': 600, '_num_ps_replicas': 0, '_keep_checkpoint_max': 5, '_tf_random_seed': None, '_task_type': None, '_environment': 'local', '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fd5e98bf3d0>, '_tf_config': gpu_options {\n", " per_process_gpu_memory_fraction: 1.0\n", "}\n", ", '_task_id': 0, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_evaluation_master': '', '_keep_checkpoint_every_n_hours': 10000, '_master': ''}\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/monitors.py:322: __init__ (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.\n", "Instructions for updating:\n", "Monitors are deprecated. Please use tf.train.SessionRunHook.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/monitors.py:322: __init__ (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.\n", "Instructions for updating:\n", "Monitors are deprecated. Please use tf.train.SessionRunHook.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/head.py:1362: scalar_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30.\n", "Instructions for updating:\n", "Please switch to tf.summary.scalar. Note that tf.summary.scalar uses the node name instead of the tag. This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. Also, passing a tensor or list of tags to a scalar summary op is no longer supported.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/head.py:1362: scalar_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30.\n", "Instructions for updating:\n", "Please switch to tf.summary.scalar. Note that tf.summary.scalar uses the node name instead of the tag. This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. Also, passing a tensor or list of tags to a scalar summary op is no longer supported.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Create CheckpointSaverHook.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Create CheckpointSaverHook.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Saving checkpoints for 1 into ./taxi_trained/train/model.ckpt.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Saving checkpoints for 1 into ./taxi_trained/train/model.ckpt.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 179.78, step = 1\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 179.78, step = 1\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/head.py:1362: scalar_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30.\n", "Instructions for updating:\n", "Please switch to tf.summary.scalar. Note that tf.summary.scalar uses the node name instead of the tag. This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. Also, passing a tensor or list of tags to a scalar summary op is no longer supported.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/head.py:1362: scalar_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30.\n", "Instructions for updating:\n", "Please switch to tf.summary.scalar. Note that tf.summary.scalar uses the node name instead of the tag. This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. Also, passing a tensor or list of tags to a scalar summary op is no longer supported.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Starting evaluation at 2017-02-28-16:38:02\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Starting evaluation at 2017-02-28-16:38:02\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [1/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [1/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [2/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [2/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [3/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [3/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [4/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [4/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [5/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [5/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [6/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [6/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [7/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [7/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [8/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [8/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [9/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [9/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [10/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [10/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [11/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [11/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [12/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [12/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [13/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [13/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [14/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [14/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [15/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [15/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [16/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [16/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [17/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [17/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [18/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [18/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [19/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [19/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [20/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [20/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [21/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [21/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [22/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [22/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [23/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [23/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [24/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [24/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [25/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [25/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [26/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [26/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [27/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [27/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [28/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [28/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [29/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [29/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [30/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [30/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [31/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [31/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [32/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [32/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [33/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [33/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [34/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [34/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [35/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [35/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [36/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [36/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [37/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [37/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [38/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [38/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [39/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [39/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [40/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [40/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [41/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [41/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [42/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [42/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [43/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [43/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [44/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [44/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [45/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [45/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [46/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [46/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [47/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [47/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [48/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [48/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [49/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [49/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [50/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [50/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [51/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [51/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [52/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [52/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [53/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [53/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [54/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [54/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [55/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [55/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [56/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [56/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [57/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [57/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [58/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [58/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [59/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [59/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [60/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [60/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [61/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [61/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [62/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [62/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [63/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [63/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [64/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [64/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [65/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [65/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [66/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [66/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [67/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [67/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [68/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [68/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [69/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [69/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [70/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [70/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [71/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [71/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [72/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [72/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [73/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [73/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [74/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [74/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [75/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [75/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [76/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [76/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [77/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [77/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [78/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [78/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [79/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [79/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [80/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [80/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [81/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [81/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [82/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [82/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [83/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [83/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [84/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [84/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [85/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [85/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [86/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [86/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [87/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [87/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [88/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [88/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [89/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [89/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [90/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [90/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [91/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [91/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [92/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [92/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [93/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [93/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [94/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [94/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [95/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [95/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [96/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [96/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [97/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [97/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [98/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [98/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [99/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [99/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [100/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [100/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Finished evaluation at 2017-02-28-16:38:02\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Finished evaluation at 2017-02-28-16:38:02\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Saving dict for global step 1: global_step = 1, loss = 190.629\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Saving dict for global step 1: global_step = 1, loss = 190.629\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:Skipping summary for global_step, must be a float or np.float32.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:Skipping summary for global_step, must be a float or np.float32.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Validation (step 100): loss = 190.629, global_step = 1\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Validation (step 100): loss = 190.629, global_step = 1\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 74.662\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 74.662\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 69.5507, step = 101\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 69.5507, step = 101\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 137.977\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 137.977\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 118.545, step = 201\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 118.545, step = 201\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 300.414\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 300.414\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 68.7011, step = 301\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 68.7011, step = 301\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 308.763\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 308.763\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 119.304, step = 401\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 119.304, step = 401\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 309.365\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 309.365\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 59.21, step = 501\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 59.21, step = 501\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 308.573\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 308.573\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 113.4, step = 601\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 113.4, step = 601\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 299.496\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 299.496\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 51.9584, step = 701\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 51.9584, step = 701\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 316.617\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 316.617\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 88.6599, step = 801\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 88.6599, step = 801\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 304.868\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 304.868\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 113.445, step = 901\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 113.445, step = 901\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 307.353\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 307.353\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 100.481, step = 1001\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 100.481, step = 1001\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 306.9\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 306.9\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 76.0784, step = 1101\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 76.0784, step = 1101\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 157.218\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 157.218\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 115.233, step = 1201\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 115.233, step = 1201\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 303.473\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 303.473\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 94.4111, step = 1301\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 94.4111, step = 1301\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 305.12\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 305.12\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 88.9203, step = 1401\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 88.9203, step = 1401\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 302.649\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 302.649\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 45.8956, step = 1501\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 45.8956, step = 1501\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 302.118\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 302.118\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 79.2905, step = 1601\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 79.2905, step = 1601\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 305.848\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 305.848\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 94.6833, step = 1701\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 94.6833, step = 1701\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 301.63\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 301.63\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 92.6318, step = 1801\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 92.6318, step = 1801\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 309.005\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 309.005\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 141.35, step = 1901\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 141.35, step = 1901\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 305.967\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 305.967\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 71.6119, step = 2001\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 71.6119, step = 2001\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 301.216\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 301.216\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 78.3532, step = 2101\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 78.3532, step = 2101\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 299.464\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 299.464\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 52.048, step = 2201\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 52.048, step = 2201\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 302.968\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 302.968\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 56.6322, step = 2301\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 56.6322, step = 2301\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 303.306\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:global_step/sec: 303.306\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 148.738, step = 2401\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:loss = 148.738, step = 2401\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Saving checkpoints for 2500 into ./taxi_trained/train/model.ckpt.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Saving checkpoints for 2500 into ./taxi_trained/train/model.ckpt.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Loss for final step: 93.5237.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Loss for final step: 93.5237.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/head.py:1362: scalar_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30.\n", "Instructions for updating:\n", "Please switch to tf.summary.scalar. Note that tf.summary.scalar uses the node name instead of the tag. This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. Also, passing a tensor or list of tags to a scalar summary op is no longer supported.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/head.py:1362: scalar_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30.\n", "Instructions for updating:\n", "Please switch to tf.summary.scalar. Note that tf.summary.scalar uses the node name instead of the tag. This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. Also, passing a tensor or list of tags to a scalar summary op is no longer supported.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Starting evaluation at 2017-02-28-16:38:11\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Starting evaluation at 2017-02-28-16:38:11\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [1/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [1/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [2/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [2/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [3/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [3/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [4/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [4/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [5/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [5/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [6/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [6/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [7/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [7/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [8/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [8/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [9/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [9/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [10/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [10/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [11/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [11/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [12/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [12/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [13/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [13/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [14/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [14/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [15/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [15/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [16/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [16/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [17/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [17/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [18/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [18/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [19/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [19/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [20/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [20/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [21/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [21/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [22/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [22/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [23/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [23/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [24/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [24/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [25/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [25/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [26/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [26/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [27/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [27/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [28/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [28/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [29/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [29/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [30/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [30/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [31/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [31/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [32/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [32/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [33/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [33/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [34/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [34/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [35/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [35/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [36/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [36/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [37/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [37/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [38/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [38/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [39/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [39/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [40/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [40/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [41/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [41/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [42/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [42/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [43/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [43/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [44/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [44/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [45/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [45/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [46/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [46/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [47/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [47/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [48/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [48/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [49/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [49/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [50/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [50/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [51/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [51/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [52/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [52/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [53/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [53/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [54/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [54/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [55/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [55/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [56/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [56/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [57/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [57/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [58/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [58/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [59/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [59/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [60/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [60/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [61/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [61/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [62/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [62/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [63/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [63/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [64/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [64/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [65/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [65/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [66/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [66/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [67/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [67/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [68/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [68/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [69/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [69/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [70/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [70/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [71/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [71/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [72/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [72/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [73/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [73/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [74/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [74/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [75/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [75/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [76/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [76/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [77/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [77/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [78/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [78/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [79/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [79/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [80/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [80/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [81/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [81/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [82/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [82/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [83/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [83/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [84/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [84/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [85/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [85/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [86/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [86/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [87/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [87/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [88/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [88/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [89/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [89/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [90/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [90/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [91/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [91/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [92/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [92/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [93/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [93/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [94/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [94/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [95/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [95/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [96/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [96/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [97/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [97/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [98/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [98/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [99/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [99/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [100/100]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Evaluation [100/100]\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Finished evaluation at 2017-02-28-16:38:12\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Finished evaluation at 2017-02-28-16:38:12\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Saving dict for global step 2500: global_step = 2500, loss = 84.0451\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Saving dict for global step 2500: global_step = 2500, loss = 84.0451\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:Skipping summary for global_step, must be a float or np.float32.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING:tensorflow:Skipping summary for global_step, must be a float or np.float32.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets added to graph.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Assets added to graph.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:No assets to write.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:No assets to write.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:SavedModel written to: ./taxi_trained/train/export/intermediate_evaluation_models/1488299892834/saved_model.pb\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:SavedModel written to: ./taxi_trained/train/export/intermediate_evaluation_models/1488299892834/saved_model.pb\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets added to graph.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Assets added to graph.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:No assets to write.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:No assets to write.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:SavedModel written to: ./taxi_trained/train/export/intermediate_prediction_models/1488299893441/saved_model.pb\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:SavedModel written to: ./taxi_trained/train/export/intermediate_prediction_models/1488299893441/saved_model.pb\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Local training done.\n" ] } ], "source": [ "eval_csv = ml.CsvDataSet(\n", " file_pattern=os.path.join(INDIR, 'valid*'),\n", " schema_file=os.path.join(OUTDIR, 'taxifare.json'))\n", "\n", "shutil.rmtree(os.path.join(OUTDIR, 'taxi_trained'), ignore_errors=True)\n", "sd.local_train(\n", " train_dataset=train_csv,\n", " eval_dataset=eval_csv,\n", " preprocess_output_dir=os.path.join(OUTDIR, 'taxi_preproc'),\n", " transforms=os.path.join(OUTDIR, 'taxi_preproc/transforms.json'),\n", " output_dir=os.path.join(OUTDIR, 'taxi_trained'),\n", " model_type='dnn_regression',\n", " max_steps=2500,\n", " layer_sizes=[64, 4]\n", ")" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "evaluation_model model train\r\n" ] } ], "source": [ "!ls taxi_trained" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Starting local prediction.\n", "Local prediction done.\n" ] }, { "data": { "text/html": [ "<div>\n", "<table border=\"1\" class=\"dataframe\">\n", " <thead>\n", " <tr style=\"text-align: right;\">\n", " <th></th>\n", " <th>key_from_input</th>\n", " <th>predicted_target</th>\n", " </tr>\n", " </thead>\n", " <tbody>\n", " <tr>\n", " <th>0</th>\n", " <td>row_01</td>\n", " <td>11.724672</td>\n", " </tr>\n", " <tr>\n", " <th>1</th>\n", " <td>row_02</td>\n", " <td>11.658803</td>\n", " </tr>\n", " <tr>\n", " <th>2</th>\n", " <td>row_03</td>\n", " <td>11.299863</td>\n", " </tr>\n", " <tr>\n", " <th>3</th>\n", " <td>row_04</td>\n", " <td>11.723350</td>\n", " </tr>\n", " <tr>\n", " <th>4</th>\n", " <td>row_05</td>\n", " <td>11.683121</td>\n", " </tr>\n", " <tr>\n", " <th>5</th>\n", " <td>row_06</td>\n", " <td>11.679772</td>\n", " </tr>\n", " <tr>\n", " <th>6</th>\n", " <td>row_07</td>\n", " <td>11.680917</td>\n", " </tr>\n", " <tr>\n", " <th>7</th>\n", " <td>row_08</td>\n", " <td>11.680292</td>\n", " </tr>\n", " <tr>\n", " <th>8</th>\n", " <td>row_09</td>\n", " <td>11.678738</td>\n", " </tr>\n", " <tr>\n", " <th>9</th>\n", " <td>row_10</td>\n", " <td>11.677299</td>\n", " </tr>\n", " </tbody>\n", "</table>\n", "</div>" ], "text/plain": [ " key_from_input predicted_target\n", "0 row_01 11.724672\n", "1 row_02 11.658803\n", "2 row_03 11.299863\n", "3 row_04 11.723350\n", "4 row_05 11.683121\n", "5 row_06 11.679772\n", "6 row_07 11.680917\n", "7 row_08 11.680292\n", "8 row_09 11.678738\n", "9 row_10 11.677299" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sd.local_predict(\n", " training_ouput_dir=os.path.join(OUTDIR, 'taxi_trained'),\n", " data=['Sun,0,-73.984685,40.769262,-73.991065,40.728145,5.0,row_01',\n", " 'Mon,12,-74.006927,40.739993,-73.950025,40.773403,1.0,row_02',\n", " 'Tues,8,-73.977345,40.779387,-73.97615,40.778867,1.0,row_03',\n", " 'Fri,17,-73.97136,40.794413,-73.99623,40.74524,1.0,row_04',\n", " 'Sun,0,-73.997642,40.763853,-73.99485,40.750282,1.0,row_05',\n", " 'Sun,0,-74.004538,40.742202,-73.955823,40.773485,1.0,row_06',\n", " 'Sun,0,-74.000589,40.73731,-73.985902,40.692725,1.0,row_07',\n", " 'Sun,0,-73.995432,40.72114,-73.992403,40.719745,1.0,row_08',\n", " 'Sun,0,-73.945033,40.779203,-73.952037,40.766802,1.0,row_09',\n", " 'Sun,0,-73.968592,40.693262,-73.99231,40.694317,1.0,row_10']\n", ")" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Starting local batch prediction.\n", "Local batch prediction done.\n" ] } ], "source": [ "shutil.rmtree(os.path.join(OUTDIR,'batch_predict'), ignore_errors=True)\n", "sd.local_batch_predict(\n", " training_ouput_dir=os.path.join(OUTDIR, 'taxi_trained'),\n", " prediction_input_file=os.path.join(INDIR, 'valid*'),\n", " output_dir=os.path.join(OUTDIR, 'batch_predict'),\n", " output_format='csv',\n", " mode='evaluation' # mode=prediction if you don't have target column\n", ")" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2009-04-28 01:01:23.000000-73.980640.750940.7533-73.9696,11.4058189392,4.90000009537\r\n", "2009-10-27 02:46:24.000000-73.980840.758840.7605-73.9931,11.1602220535,4.90000009537\r\n", "2010-10-05 02:09:00.000000-74.000140.730240.7526-74.0039,11.1707496643,7.69999980927\r\n", "2012-05-08 03:50:00.000000-74.003840.73840.7591-73.9904,11.6874761581,6.90000009537\r\n", "2012-05-08 03:50:00.000000-73.988840.736540.7193-73.9436,11.727016449,12.1000003815\r\n", "2012-12-04 06:52:02.000000-73.982240.731840.7725-73.951,11.7066316605,11.0\r\n", "2012-11-20 06:53:30.000000-73.94740.771940.7554-73.9759,11.7086172104,9.0\r\n", "2014-05-13 07:33:52.000000-73.962340.79540.7853-73.973,10.7292032242,5.0\r\n", "2011-05-17 07:53:55.000000-73.970540.752340.7467-73.9817,10.7170114517,5.30000019073\r\n", "2012-10-09 07:47:46.000000-73.973940.747740.7256-74.0055,10.7183790207,13.5\r\n" ] } ], "source": [ "!head $OUTDIR/batch_predict/predictions-00000-*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<h2> Cloud preprocessing and training </h2>\n", "\n", "In the above cells, change INDIR and OUTDIR to be GCS.\n", "\n", "Change the calls from local_predict to cloud_predict. That's it.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Copyright 2016 Google Inc. Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.5" } }, "nbformat": 4, "nbformat_minor": 2 }

courses/machine_learning/packaged/sdsoln.ipynb (4,508 lines of code) (raw):