baseline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Предскажите зарплату по тексту объявления\n",
    "\n",
    "На сайтах для поиска работы можно найти сотни тысяч объявлений, каждое из которых состоит из пространного описания вакансии и предлагаемой зарплаты. Есть ли связь между описанием и зарплатой? Существуют ли определенные слова, которые наиболее сильно характеризуют зарплату? Можно ли найти другие информативные факторы? Вам предстоит ответить на эти вопросы, проанализировав выборку объявлений о работе в Великобритании. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "%pylab inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Introduction.ipynb baseline.ipynb     features-train.zip\r\n",
      "adult.data         features-train.csv intro.ipynb\r\n"
     ]
    }
   ],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Разархивируем скачанный архив с помощью консольной команды unzip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UnZip 5.52 of 28 February 2005, by Info-ZIP.  Maintained by C. Spieler.  Send\r\n",
      "bug reports using http://www.info-zip.org/zip-bug.html; see README for details.\r\n",
      "\r\n",
      "Usage: unzip [-Z] [-opts[modifiers]] file[.zip] [list] [-x xlist] [-d exdir]\r\n",
      "  Default action is to extract files in list, except those in xlist, to exdir;\r\n",
      "  file[.zip] may be a wildcard.  -Z => ZipInfo mode (\"unzip -Z\" for usage).\r\n",
      "\r\n",
      "  -p  extract files to pipe, no messages     -l  list files (short format)\r\n",
      "  -f  freshen existing files, create none    -t  test compressed archive data\r\n",
      "  -u  update files, create if necessary      -z  display archive comment\r\n",
      "  -x  exclude files that follow (in xlist)   -d  extract files into exdir\r\n",
      "\r\n",
      "modifiers:                                   -q  quiet mode (-qq => quieter)\r\n",
      "  -n  never overwrite existing files         -a  auto-convert any text files\r\n",
      "  -o  overwrite files WITHOUT prompting      -aa treat ALL files as text\r\n",
      "  -j  junk paths (do not make directories)   -v  be verbose/print version info\r\n",
      "  -C  match filenames case-insensitively     -L  make (some) names lowercase\r\n",
      "  -X  restore UID/GID info                   -V  retain VMS version numbers\r\n",
      "  -K  keep setuid/setgid/tacky permissions   -M  pipe through \"more\" pager\r\n",
      "Examples (see unzip.txt for more info):\r\n",
      "  unzip data1 -x joe   => extract all files except joe from zipfile data1.zip\r\n",
      "  unzip -p foo | more  => send contents of foo.zip via pipe into program more\r\n",
      "  unzip -fo foo ReadMe => quietly replace existing ReadMe if archive file newer\r\n"
     ]
    }
   ],
   "source": [
    "!unzip --help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Archive:  features-train.zip\n",
      "  inflating: features-train.csv      \n"
     ]
    }
   ],
   "source": [
    "!unzip features-train.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Introduction.ipynb baseline.ipynb     features-train.zip\r\n",
      "adult.data         features-train.csv intro.ipynb\r\n"
     ]
    }
   ],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Появился файл features-train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Объявляем используемые библиотеки"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import re\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Id,Title,FullDescription,LocationRaw,LocationNormalized,ContractType,ContractTime,Company,Category,SalaryRaw,SalaryNormalized,SourceName\r\n",
      "0,Chartered Senior Quantity Surveyor,\"Award winning multi disciplinary consultancy are looking for an experienced MRICS/FRICS Senior Quantity Surveyor to join their successful growing Quantity Surveying team. This is an excellent opportunity for an experienced Chartered Quantity Surveyor to take on team management responsibilities whilst maintaining a hands on involvement in providing and growing the Quantity Surveying team in the South West area. You will be highly involved in business development and will work alongside the Director on all cost related projects. You will be managing a small team of Quantity Surveyors mentoring APC junior Quantity Surveyors and guiding Chartered QS on a variety of projects from Pre contract  Post Contract. The successful candidate will need to be chartered MRICS/FRICS Senior Quantity Surveyor with consultancy experience. You will need be a client facing Senior Quantity Surveyor who has a proven track record in business development & has experience of growing a QS division. Candidates must be dynamic, ambitious & enthusiastic with good interpersonal & business development skills. You will need to have a broad range of sector experience and have proven consultancy experience delivering projects in Education, Offices, Retail, & Commercial sector, Some experience working with Housing Associations sector would be advantageous but not essential. If you feel you have the relevant experience and would like to discuss this fantastic opportunity in more detail please call Lee Faux  (Apply online only) or email\",Somerset,Somerset,,permanent,Resourcing Group,Consultancy Jobs,45000 - 55000/annum,50000,cv-library.co.uk\r\n"
     ]
    }
   ],
   "source": [
    "!head -n 2 features-train.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>FullDescription</th>\n",
       "      <th>LocationRaw</th>\n",
       "      <th>LocationNormalized</th>\n",
       "      <th>ContractType</th>\n",
       "      <th>ContractTime</th>\n",
       "      <th>Company</th>\n",
       "      <th>Category</th>\n",
       "      <th>SalaryRaw</th>\n",
       "      <th>SalaryNormalized</th>\n",
       "      <th>SourceName</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Chartered Senior Quantity Surveyor</td>\n",
       "      <td>Award winning multi disciplinary consultancy a...</td>\n",
       "      <td>Somerset</td>\n",
       "      <td>Somerset</td>\n",
       "      <td>NaN</td>\n",
       "      <td>permanent</td>\n",
       "      <td>Resourcing Group</td>\n",
       "      <td>Consultancy Jobs</td>\n",
       "      <td>45000 - 55000/annum</td>\n",
       "      <td>50000</td>\n",
       "      <td>cv-library.co.uk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Net Developer  VB / SQL  Central Manchester  *...</td>\n",
       "      <td>A senior / experienced VB.Net Developer, with ...</td>\n",
       "      <td>Manchester, Greater Manchester, England, Great...</td>\n",
       "      <td>Manchester</td>\n",
       "      <td>NaN</td>\n",
       "      <td>permanent</td>\n",
       "      <td>CV Screen Ltd</td>\n",
       "      <td>IT Jobs</td>\n",
       "      <td>30000 - 45000/annum + benefits</td>\n",
       "      <td>37500</td>\n",
       "      <td>cv-library.co.uk</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Desktop Support Administrator  Windows 7, Exch...</td>\n",
       "      <td>Desktop Support Administrator  Windows 7, Exch...</td>\n",
       "      <td>Reading Berkshire South East</td>\n",
       "      <td>Reading</td>\n",
       "      <td>NaN</td>\n",
       "      <td>permanent</td>\n",
       "      <td>Spectrum IT</td>\n",
       "      <td>IT Jobs</td>\n",
       "      <td>26000 - 30000 per annum + excellent benefits</td>\n",
       "      <td>28000</td>\n",
       "      <td>totaljobs.com</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Clinical Lead (RGN)</td>\n",
       "      <td>RGN Clinical Lead Mid Glamorgan Very competiti...</td>\n",
       "      <td>Merthyr Tydfil Mid Glamorgan Wales</td>\n",
       "      <td>UK</td>\n",
       "      <td>NaN</td>\n",
       "      <td>permanent</td>\n",
       "      <td>BROOK STREET BUREAU   Cardiff Care</td>\n",
       "      <td>Healthcare &amp; Nursing Jobs</td>\n",
       "      <td>26026 - 29029 per annum + Benefits</td>\n",
       "      <td>27527</td>\n",
       "      <td>totaljobs.com</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Charity Fundraisers  GET PAID TO SAVE LIVES</td>\n",
       "      <td>CHARITY FUNDRAISING REPRESENTATIVES  GET PAID ...</td>\n",
       "      <td>Exeter,Devon,EXMOUTH,CREDITON,OTTERY ST MARY,D...</td>\n",
       "      <td>UK</td>\n",
       "      <td>NaN</td>\n",
       "      <td>permanent</td>\n",
       "      <td>Recruit4Talent</td>\n",
       "      <td>Sales Jobs</td>\n",
       "      <td>7 per hour + bonus</td>\n",
       "      <td>13440</td>\n",
       "      <td>jobsite.co.uk</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Title  \\\n",
       "Id                                                      \n",
       "0                  Chartered Senior Quantity Surveyor   \n",
       "1   Net Developer  VB / SQL  Central Manchester  *...   \n",
       "2   Desktop Support Administrator  Windows 7, Exch...   \n",
       "3                                 Clinical Lead (RGN)   \n",
       "4         Charity Fundraisers  GET PAID TO SAVE LIVES   \n",
       "\n",
       "                                      FullDescription  \\\n",
       "Id                                                      \n",
       "0   Award winning multi disciplinary consultancy a...   \n",
       "1   A senior / experienced VB.Net Developer, with ...   \n",
       "2   Desktop Support Administrator  Windows 7, Exch...   \n",
       "3   RGN Clinical Lead Mid Glamorgan Very competiti...   \n",
       "4   CHARITY FUNDRAISING REPRESENTATIVES  GET PAID ...   \n",
       "\n",
       "                                          LocationRaw LocationNormalized  \\\n",
       "Id                                                                         \n",
       "0                                            Somerset           Somerset   \n",
       "1   Manchester, Greater Manchester, England, Great...         Manchester   \n",
       "2                        Reading Berkshire South East            Reading   \n",
       "3                  Merthyr Tydfil Mid Glamorgan Wales                 UK   \n",
       "4   Exeter,Devon,EXMOUTH,CREDITON,OTTERY ST MARY,D...                 UK   \n",
       "\n",
       "   ContractType ContractTime                             Company  \\\n",
       "Id                                                                 \n",
       "0           NaN    permanent                    Resourcing Group   \n",
       "1           NaN    permanent                       CV Screen Ltd   \n",
       "2           NaN    permanent                         Spectrum IT   \n",
       "3           NaN    permanent  BROOK STREET BUREAU   Cardiff Care   \n",
       "4           NaN    permanent                      Recruit4Talent   \n",
       "\n",
       "                     Category                                     SalaryRaw  \\\n",
       "Id                                                                            \n",
       "0            Consultancy Jobs                           45000 - 55000/annum   \n",
       "1                     IT Jobs                30000 - 45000/annum + benefits   \n",
       "2                     IT Jobs  26000 - 30000 per annum + excellent benefits   \n",
       "3   Healthcare & Nursing Jobs            26026 - 29029 per annum + Benefits   \n",
       "4                  Sales Jobs                            7 per hour + bonus   \n",
       "\n",
       "    SalaryNormalized        SourceName  \n",
       "Id                                      \n",
       "0              50000  cv-library.co.uk  \n",
       "1              37500  cv-library.co.uk  \n",
       "2              28000     totaljobs.com  \n",
       "3              27527     totaljobs.com  \n",
       "4              13440     jobsite.co.uk  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv('features-train.csv', header=0, index_col=0, sep=',')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "all_texts = data[filter(lambda c: 'Salary' not in c, data.columns)].values\n",
    "all_texts = map(lambda a: re.sub('[^a-z]', ' ', ' '.join(a).lower()), all_texts.astype(str))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "179999"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(all_texts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'chartered senior quantity surveyor award winning multi disciplinary consultancy are looking for an experienced mrics frics senior quantity surveyor to join their successful growing quantity surveying team  this is an excellent opportunity for an experienced chartered quantity surveyor to take on team management responsibilities whilst maintaining a hands on involvement in providing and growing the quantity surveying team in the south west area  you will be highly involved in business development and will work alongside the director on all cost related projects  you will be managing a small team of quantity surveyors mentoring apc junior quantity surveyors and guiding chartered qs on a variety of projects from pre contract  post contract  the successful candidate will need to be chartered mrics frics senior quantity surveyor with consultancy experience  you will need be a client facing senior quantity surveyor who has a proven track record in business development   has experience of growing a qs division  candidates must be dynamic  ambitious   enthusiastic with good interpersonal   business development skills  you will need to have a broad range of sector experience and have proven consultancy experience delivering projects in education  offices  retail    commercial sector  some experience working with housing associations sector would be advantageous but not essential  if you feel you have the relevant experience and would like to discuss this fantastic opportunity in more detail please call lee faux   apply online only  or email somerset somerset nan permanent resourcing group consultancy jobs cv library co uk'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_texts[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
    "\n",
    "vectorizer = TfidfVectorizer(stop_words='english', min_df=5)\n",
    "_ = vectorizer.fit(all_texts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X = vectorizer.transform(all_texts)\n",
    "Y = data.SalaryNormalized.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ridge regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import ElasticNet, Ridge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 16 s, sys: 131 ms, total: 16.2 s\n",
      "Wall time: 16.3 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "lr = Ridge(solver='sag', alpha=0.1)\n",
    "lr.fit(X,Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import r2_score, mean_absolute_error, mean_squared_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2 0.78\n",
      "MAE 5798.47\n",
      "RMSE 8311.72\n"
     ]
    }
   ],
   "source": [
    "print 'R2 %.2f' % r2_score(Y, lr.predict(X))\n",
    "print 'MAE %.2f' % mean_absolute_error(Y, lr.predict(X))\n",
    "print 'RMSE %.2f' % np.sqrt(mean_squared_error(Y, lr.predict(X)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA8IAAAHfCAYAAACbPGHBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+wZnddJ/j3B1KsO4KBZASrg95YC0R0tbLUbCjHLcxO\nHND9Q1hdKRa3SNZohqFZ3XVrR5i2xGGgkGxpoWujhWY1WLIpxqlpcIeCyI87lLuguZomQrBpXPtK\ncps2E+wU10B30vnuH/dpeAjdz7npvv187z3n9apKec95znPv90aefO77Oed9nmqtBQAAAKbiSb0X\nAAAAAMskCAMAADApgjAAAACTIggDAAAwKYIwAAAAkyIIAwAAMCmDQbiqnl1VH66qT1XVX1TV/zTb\n/4aquq+q/nz2zw/OPef1VXW0qj5dVS+e2/+Cqrqnqj5TVW+b2/+Uqrpj9pyPVdW37fQvCgAAAMn2\nzgg/muRnW2vfleR7k7y2qr5j9tivtNZeMPvn/UlSVc9P8vIkz0/yQ0neXlU1O/43ktzcWntekudV\n1Utm+29O8oXW2nOTvC3JrTvxywEAAMDjDQbh1trnW2uHZ19vJvl0kqtmD9c5nvLSJHe01h5trR1L\ncjTJdVX1LUme1lq7a3bcO5O8bO45t8++/oMkN1zA7wIAAACDnlBHuKquTnJtkj+Z7XptVR2uqt+u\nqstn+65K8rm5p90/23dVkvvm9t+XrwbqrzyntXYmycmquuKJrA0AAAC247LtHlhVT83W2dqfaa1t\nVtXbk7yxtdaq6k1JfjnJT+7Qus51pjlV1Xbo+wMAALALtdbOmQd30raCcFVdlq0Q/HuttfckSWvt\ngblDfivJH86+vj/Jt8499uzZvvPtn3/ORlU9Ock3tda+cK61tCYLw7L94i/+Yn7xF3+x9zJgcrz2\noB+vP+jjq7eXurS2e2n0/5nk3tbar57dMev8nvUjST45+/q9SV4xuxP0tyd5TpI/ba19PslDVXXd\n7OZZr0rynrnn3Dj7+seSfPiCfhvgkjh27FjvJcAkee1BP15/MG6DZ4Sr6vuS/HiSv6iqu5O0JP8y\nySur6tokjyU5luSfJUlr7d6qeneSe5M8kuQ17auncfcn+d0k35DkfWfvNJ3ktiS/V1VHkzyY5BU7\n8tsBAADA49ReutS4qtpeWi+Mxerqaq6//vrey4DJ8dqDfrz+oI+qWkpHWBAGAABgV1hWEH5CH58E\nTNPq6mrvJcAkee1BP15/MG6CMAAAAJPi0mgAAAB2BZdGAwAAwCUw+PFJAO6cCZfGwYPvysbG5nkf\n39z8m/zqr75piSsCzjL7YNwEYQDoZGNjMysrt5z38Y9+9H9d4moAYDpcGg0M8o449LGyck3vJcBk\nmX0wboIwAAAAkyIIA4N8liL0sb5+pPcSYLLMPhg3QRgAAIBJEYSBQXpS0IeOMPRj9sG4CcIAAABM\niiAMDNKTgj50hKEfsw/GTRAGAABgUgRhYJCeFPShIwz9mH0wboIwAAAAkyIIA4P0pKAPHWHox+yD\ncROEAQAAmBRBGBikJwV96AhDP2YfjJsgDAAAwKQIwsAgPSnoQ0cY+jH7YNwEYQAAACZFEAYG6UlB\nHzrC0I/ZB+MmCAMAADApgjAwSE8K+tARhn7MPhg3QRgAAIBJEYSBQXpS0IeOMPRj9sG4CcIAAABM\niiAMDNKTgj50hKEfsw/GTRAGAABgUgRhYJCeFPShIwz9mH0wbpf1XgAAcG5ra4dz4MA7Fh6zb99T\ns3//K5e0IgAYB0EYGLS6uuqdcejg/vvvy4/+6NsXHrO+vjgoAxfG7INxc2k0AAAAkyIIA4O8Iw59\nXH75vt5LgMky+2DcBGEAAAAmRRAGBvksRejjoYc2ei8BJsvsg3EThAEAAJgUQRgYpCcFfegIQz9m\nH4ybIAwAAMCkCMLAID0p6ENHGPox+2DcBGEAAAAmRRAGBulJQR86wtCP2QfjJggDAAAwKYIwMEhP\nCvrQEYZ+zD4YN0EYAACASRGEgUF6UtCHjjD0Y/bBuAnCAAAATIogDAzSk4I+dIShH7MPxk0QBgAA\nYFIEYWCQnhT0oSMM/Zh9MG6CMAAAAJMiCAOD9KSgDx1h6Mfsg3EThAEAAJgUQRgYpCcFfegIQz9m\nH4ybIAwAAMCkCMLAID0p6ENHGPox+2DcBGEAAAAmRRAGBulJQR86wtCP2QfjJggDAAAwKYIwMEhP\nCvrQEYZ+zD4YN0EYAACASbms9wKA3U9PCp64gwfflY2NzYXHrK19Misr539cRxj6Mftg3ARhALgE\nNjY2s7Jyy8JjVldfvaTVAADzXBoNDNKTgj50hKEfsw/GTRAGAABgUgRhYJCeFPShIwz9mH0wboIw\nAAAAkyIIA4P0pKAPHWHox+yDcROEAQAAmBRBGBikJwV96AhDP2YfjJsgDAAAwKQIwsAgPSnoQ0cY\n+jH7YNwEYQAAACZFEAYG6UlBHzrC0I/ZB+MmCAMAADApgjAwSE8K+tARhn7MPhg3QRgAAIBJEYSB\nQXpS0IeOMPRj9sG4DQbhqnp2VX24qj5VVX9RVT892/+Mqrqzqo5U1Qeq6vK557y+qo5W1aer6sVz\n+19QVfdU1Weq6m1z+59SVXfMnvOxqvq2nf5FAQAAINneGeFHk/xsa+27knxvkv1V9R1JXpfkg621\na5J8OMnrk6SqvjPJy5M8P8kPJXl7VdXse/1Gkptba89L8ryqesls/81JvtBae26StyW5dUd+O2BH\n6ElBHzrC0I/ZB+M2GIRba59vrR2efb2Z5NNJnp3kpUlunx12e5KXzb7+4SR3tNYeba0dS3I0yXVV\n9S1JntZau2t23DvnnjP/vf4gyQ0X80sBAADA+TyhjnBVXZ3k2iQfT/Ks1tqJZCssJ3nm7LCrknxu\n7mn3z/ZdleS+uf33zfZ9zXNaa2eSnKyqK57I2oBLR08K+tARhn7MPhi3bQfhqnpqts7W/szszHB7\n3CGP374YNXwIAAAAPHGXbeegqrosWyH491pr75ntPlFVz2qtnZhd9vy3s/33J/nWuac/e7bvfPvn\nn7NRVU9O8k2ttS+cay033XRTrr766iTJ05/+9Fx77bVfecfubJfDtm3bO7s935PaDeuxbXsvbK+v\nH8nKSpIkR45sPX7NNdd/zfZZ53v8bEf4fI+f3d4Nv69t22PbPrtvt6zHtu2xbh8+fDgnT55Mkhw7\ndizLUq0Nn8itqncm+Y+ttZ+d2/fWbN3g6q1V9XNJntFae93sZlm/n+SF2brk+Y+SPLe11qrq40l+\nOsldSf59kl9rrb2/ql6T5D9vrb2mql6R5GWttVecYx1tO+sFdtbq6upX/oMFbM+BA+/IysotC4+5\n/fZX58Ybf/O8j//ar/1wfvqn37vwe6yvvyNvfvPinwM8cWYf9FFVaa1d8iuEB88IV9X3JfnxJH9R\nVXdn6xLof5nkrUneXVU/kWQ9W3eKTmvt3qp6d5J7kzyS5DVz6XV/kt9N8g1J3tdae/9s/21Jfq+q\njiZ5MMnXhWCgH38IQB86wtCP2QfjNhiEW2v/T5Inn+fhHzjPc96S5C3n2P9nSb77HPtPZRakAQAA\n4FJ6Uu8FALvffF8KWB6fIwz9mH0wboIwAAAAkyIIA4P0pKAPHWHox+yDcROEAQAAmBRBGBikJwV9\n6AhDP2YfjJsgDAAAwKQIwsAgPSnoQ0cY+jH7YNwEYQAAACZFEAYG6UlBHzrC0I/ZB+MmCAMAADAp\ngjAwSE8K+tARhn7MPhg3QRgAAIBJEYSBQXpS0IeOMPRj9sG4CcIAAABMiiAMDNKTgj50hKEfsw/G\nTRAGAABgUgRhYJCeFPShIwz9mH0wboIwAAAAkyIIA4P0pKAPHWHox+yDcROEAQAAmBRBGBikJwV9\n6AhDP2YfjJsgDAAAwKQIwsAgPSnoQ0cY+jH7YNwEYQAAACZFEAYG6UlBHzrC0I/ZB+N2We8FAAAX\nbm3tcA4ceMfCY/bte2r273/lklYEALufIAwM0pOCPrbTEd7cfCwrK7csPGZ9fXFQBr6e2Qfj5tJo\nAAAAJkUQBgbpSUEfOsLQj9kH4yYIAwAAMCmCMDBITwr68DnC0I/ZB+MmCAMAADApgjAwSE8K+tAR\nhn7MPhg3QRgAAIBJEYSBQXpS0IeOMPRj9sG4CcIAAABMiiAMDNKTgj50hKEfsw/GTRAGAABgUgRh\nYJCeFPShIwz9mH0wboIwAAAAkyIIA4P0pKAPHWHox+yDcROEAQAAmBRBGBikJwV96AhDP2YfjJsg\nDAAAwKQIwsAgPSnoQ0cY+jH7YNwEYQAAACZFEAYG6UlBHzrC0I/ZB+MmCAMAADApgjAwSE8K+tAR\nhn7MPhg3QRgAAIBJEYSBQXpS0IeOMPRj9sG4CcIAAABMiiAMDNKTgj50hKEfsw/GTRAGAABgUgRh\nYJCeFPShIwz9mH0wboIwAAAAkyIIA4P0pKAPHWHox+yDcROEAQAAmBRBGBikJwV96AhDP2YfjJsg\nDAAAwKQIwsAgPSnoQ0cY+jH7YNwEYQAAACZFEAYG6UlBHzrC0I/ZB+N2We8FAMBec/Dgu7Kxsbnw\nmLW1T2ZlZUkLAgCeEEEYGLS6uuqdcZizsbGZlZVbFh6zuvrqi/45OsLQj9kH4+bSaAAAACZFEAYG\neUcc+tARhn7MPhg3QRgAAIBJEYSBQT5LEfrQEYZ+zD4YN0EYAACASRGEgUF6UtCHjjD0Y/bBuAnC\nAAAATIogDAzSk4I+dIShH7MPxk0QBgAAYFIEYWCQnhT0oSMM/Zh9MG6CMAAAAJMiCAOD9KSgDx1h\n6Mfsg3EThAEAAJgUQRgYpCcFfegIQz9mH4ybIAwAAMCkCMLAID0p6ENHGPox+2DcBoNwVd1WVSeq\n6p65fW+oqvuq6s9n//zg3GOvr6qjVfXpqnrx3P4XVNU9VfWZqnrb3P6nVNUds+d8rKq+bSd/QQAA\nAJi3nTPCv5PkJefY/yuttRfM/nl/klTV85O8PMnzk/xQkrdXVc2O/40kN7fWnpfkeVV19nvenOQL\nrbXnJnlbklsv/NcBLgU9KehDRxj6Mftg3AaDcGvtj5P83TkeqnPse2mSO1prj7bWjiU5muS6qvqW\nJE9rrd01O+6dSV4295zbZ1//QZIbtr98AAAAeGIupiP82qo6XFW/XVWXz/ZdleRzc8fcP9t3VZL7\n5vbfN9v3Nc9prZ1JcrKqrriIdQE7TE8K+tARhn7MPhi3yy7weW9P8sbWWquqNyX55SQ/uUNrOteZ\n5q+46aabcvXVVydJnv70p+faa6/9yqUrZ/+DZdu2bdu2bV/K7fX1I/nyl1dzzTVb20eObD0+vz0f\nYs/1+LwLfXy72+vrR7K6urpr/v3Ztr0Xts/aLeuxbXus24cPH87JkyeTJMeOHcuyVGtt+KCqlSR/\n2Fr7nkWPVdXrkrTW2ltnj70/yRuSrCf5SGvt+bP9r0jy/a21f372mNban1TVk5Mcb6098zzraNtZ\nLwBcSgcOvCMrK7csPOb221+dG2/8zYs6Zie+R5Ksr78jb37z4vUCwG5QVWmtLTw5uhOetM3jKnNn\named37N+JMknZ1+/N8krZneC/vYkz0nyp621zyd5qKqum90861VJ3jP3nBtnX/9Ykg9f0G8CAAAA\n2zAYhKvqXUn+32zd6flvqup/THLr7KOQDif5/iT/S5K01u5N8u4k9yZ5X5LXzJ3C3Z/ktiSfSXL0\n7J2mZ/v+YVUdTfI/J3ndjv12wI54/GViwHLoCEM/Zh+M22BHuLX2ynPs/p0Fx78lyVvOsf/Pknz3\nOfafytZHLgEAAMAlt91Lo4EJO3tDA2C5fI4w9GP2wbgJwgAAAEyKIAwM0pOCPnSEoR+zD8ZNEAYA\nAGBSBGFgkJ4U9KEjDP2YfTBugjAAAACTIggDg/SkoA8dYejH7INxE4QBAACYFEEYGKQnBX3oCEM/\nZh+MmyAMAADApAjCwCA9KehDRxj6Mftg3ARhAAAAJkUQBgbpSUEfOsLQj9kH4yYIAwAAMCmCMDBI\nTwr60BGGfsw+GDdBGAAAgEkRhIFBelLQh44w9GP2wbgJwgAAAEyKIAwM0pOCPnSEoR+zD8ZNEAYA\nAGBSBGFgkJ4U9KEjDP2YfTBugjAAAACTIggDg/SkoA8dYejH7INxE4QBAACYlMt6LwDY/fSkoI+d\n6givrR3OgQPvWHjMvn1Pzf79r9yRnwdjYPbBuAnCADBym5uPZWXlloXHrK8vDsoAMCYujQYG6UlB\nHzrC0I/ZB+MmCAMAADApgjAwSE8K+vA5wtCP2QfjJggDAAAwKYIwMEhPCvrQEYZ+zD4YN0EYAACA\nSRGEgUF6UtCHjjD0Y/bBuAnCAAAATIogDAzSk4I+dIShH7MPxk0QBgAAYFIEYWCQnhT0oSMM/Zh9\nMG6CMAAAAJMiCAOD9KSgDx1h6Mfsg3EThAEAAJgUQRgYpCcFfegIQz9mH4ybIAwAAMCkCMLAID0p\n6ENHGPox+2DcBGEAAAAmRRAGBulJQR86wtCP2QfjJggDAAAwKYIwMEhPCvrQEYZ+zD4YN0EYAACA\nSRGEgUF6UtCHjjD0Y/bBuAnCAAAATIogDAzSk4I+dIShH7MPxk0QBgAAYFIu670AYPfTk2JKDh58\nVzY2Nhces7b2yaysXPq16AhDP2YfjJsgDABzNjY2s7Jyy8JjVldfvaTVAACXgkujgUF6UtCHjjD0\nY/bBuAnCAAAATIogDAzSk4I+dIShH7MPxk0QBgAAYFLcLAsYtLq66p1xmHP85Cdz6CMHBo+5WDrC\n0I/ZB+MmCAPAE3Q6D+eKFy7+/KTT9zy8pNUAAE+US6OBQd4Rhz50hKEfsw/GTRAGAABgUgRhYJDP\nUoQ+dIShH7MPxk0QBgAAYFIEYWCQnhT0oSMM/Zh9MG7uGg3AKBy87WA2Hlx8KfG+K/dl/837l7Qi\nAGC3EoSBQT5Lkb1g48GNrNyw+CON1j+0vqTV7AwdYejH7INxc2k0AAAAkyIIA4O8Iw596AhDP2Yf\njJsgDAAAwKToCAOD9KSYkrVPfSR3/3+Lu8Sbpx5Yylp0hKEfsw/GTRAGYDLW7l7LgVsPLDzm6P33\n5B//+H+98Jgzdz2yk8sCAJZMEAYGeUecsdg8tTl4Z+nTd3x5SasZpiMM/Zh9MG46wgAAAEyKIAwM\nWl1d7b0EmCQdYejH7INxE4QBAACYFEEYGKQnBX3oCEM/Zh+MmyAMAADApAjCwCA9KehDRxj6Mftg\n3ARhAAAAJmUwCFfVbVV1oqrumdv3jKq6s6qOVNUHquryucdeX1VHq+rTVfXiuf0vqKp7quozVfW2\nuf1Pqao7Zs/5WFV9207+gsDF05OCPnSEoR+zD8ZtO2eEfyfJSx6373VJPthauybJh5O8Pkmq6juT\nvDzJ85P8UJK3V1XNnvMbSW5urT0vyfOq6uz3vDnJF1prz03ytiS3XsTvAwAAAAtdNnRAa+2Pq2rl\ncbtfmuT7Z1/fnmQ1W+H4h5Pc0Vp7NMmxqjqa5LqqWk/ytNbaXbPnvDPJy5J8YPa93jDb/wdJfv3C\nfx3gUlhdXfXOOJfMwdsOZuPBxV3YfVfuy/6b9y9pRbuHjjD0Y/bBuA0G4fN4ZmvtRJK01j5fVc+c\n7b8qycfmjrt/tu/RJPfN7b9vtv/scz43+15nqupkVV3RWvvCBa4NgD1k48GNrNzw+Pdbv9b6h9aX\ntBoAYAouNAg/Xtuh75MktejBm266KVdffXWS5OlPf3quvfbar7xbd/bufrZt297Z7euvv35Xrcf2\n3tn+1F99KhsPbmT9r7aC7Mp/thV457fXPrGWL1/+5STJNf/omiTJkbUjX7O9/lfrWZ07O3Oun7f+\nV+tfCdSPf/7Z7bPO9/jZ7ROf3dp+1nPOvf3ol07nxGePnPfxE589klN//8Wv/rwjW+u95prrv2Z7\n6PGzHeHzPb7d7Yce2siRI6sLjz9x4qv/jnbL/35s27Zt2/b4tw8fPpyTJ08mSY4dO5ZlqdaGM+zs\n0ug/bK19z2z700mub62dqKpvSfKR1trzq+p1SVpr7a2z496frcue188eM9v/iiTf31r752ePaa39\nSVU9Ocnx1tozv34VSVW17awXgN3hwK0HBs/23v7m23PjgRsXHrP+ofW8+V+8eSk/640/9dZc/6qf\nW3jMe379F/LS175x4TGrv/Wb+YWf+fPF67n91bnxxt887+O/9Kv/Vb7je77/vI8nyV/e8x/yup/5\n44v6OUmyvv6OvPnNtyw8BgAutapKa23hydGd8KRtHlf52jO1701y0+zrG5O8Z27/K2Z3gv72JM9J\n8qettc8neaiqrpvdPOtVj3vO2b9KfixbN98CdpGz794By/XFvz+RK164svCf03m49zJhlMw+GLfB\nS6Or6l1Jrk9yZVX9TbbO8P5Skn9TVT+RrbO9L0+S1tq9VfXuJPcmeSTJa+ZO4e5P8rtJviHJ+1pr\n75/tvy3J781urPVgklfszK8GAAAAX287d41+5Xke+oHzHP+WJG85x/4/S/Ld59h/KrMgDexOZ3sc\nwHL9J9/4tN5LgMky+2DctntpNAAAAIyCIAwM0pOCPubvPA0sl9kH4yYIAwAAMCmCMDBITwr60BGG\nfsw+GDdBGAAAgEkRhIFBelLQh44w9GP2wbgJwgAAAEyKIAwM0pOCPnSEoR+zD8ZNEAYAAGBSBGFg\nkJ4U9KEjDP2YfTBul/VeAAAMWbt7LQduPbD4mE+sZeWGlSWtCADYywRhYJCeFL1tntocDLmrH19d\nzmKWSEcY+jH7YNxcGg0AAMCkOCMMDFpdXfXOOHSwzI7w2trhHDjwjoXH7Nv31Ozf/8olrQj6Mvtg\n3ARhAOjk+MlP5tBHzt99/tIjJ5e2ls3Nx7KycsvCY9bXFwdlANgrBGFgkHfEOZeDtx3MxoMbC49x\nA6vFTufhXPHC8//7qbuevMTVAPPMPhg3QRiAC7Lx4MYkb2AFAOx9bpYFDPJZitDHo1863XsJMFlm\nH4ybIAwAAMCkCMLAID0p6OOy//QpvZcAk2X2wbgJwgAAAEyKIAwM0pOCPnSEoR+zD8ZNEAYAAGBS\nBGFgkJ4U9KEjDP2YfTBugjAAAACTIggDg/SkoA8dYejH7INxE4QBAACYFEEYGKQnBX3oCEM/Zh+M\nmyAMAADApAjCwCA9KehDRxj6Mftg3ARhAAAAJkUQBgbpSUEfOsLQj9kH43ZZ7wUAwLIcP/5ADh36\n6MJjNjcfXtJqAIBeBGFg0OrqqnfGGYXTp87kiitftPCYM499cEmrGbZTHeHjJz+ZQx85MHgM8FVm\nH4ybIAwAI3c6D+eKF64sPuYeZ8IBmA4dYWCQd8ShDx1h6Mfsg3EThAEAAJgUQRgY5LMUoQ+fIwz9\nmH0wboIwAAAAkyIIA4P0pKAPHWHox+yDcROEAQAAmBRBGBikJwV96AhDP2YfjJsgDAAAwKQIwsAg\nPSnoQ0cY+jH7YNwEYQAAACZFEAYG6UlBHzrC0I/ZB+MmCAMAADApgjAwSE8K+tARhn7MPhg3QRgA\nAIBJEYSBQXpS0IeOMPRj9sG4CcIAAABMiiAMDNKTgj50hKEfsw/G7bLeCwAALtzmww/k0EcOLD7m\n1ANLWg0A7A2CMDBodXXVO+PQwXY6wmee/EiueOHK4mPuemSnlgSTYfbBuLk0GgAAgEkRhIFB3hGH\nPnSEoR+zD8ZNEAYAAGBSBGFgkM9ShD58jjD0Y/bBuLlZFgBf5+BtB7Px4MbCY9Y+sZaVGxbfpAkA\nYDcShIFBelLTs/HgxmDIXf346nIWM2E6wtCP2Qfj5tJoAAAAJkUQBgbpSUEfOsLQj9kH4+bSaABg\nx2ynX77vyn3Zf/P+Ja0IAL6eIAwM0pOCPvZiR3g7/fL1D60vaTVw4cw+GDeXRgMAADApgjAwSE8K\n+tARhn7MPhg3QRgAAIBJEYSBQXpS0Mde7AjDWJh9MG6CMAAAAJMiCAOD9KSgDx1h6Mfsg3EThAEA\nAJgUnyMMDNKTgj52W0d4be1wDhx4x+JjPvupwc8Rhr3A7INxE4QBgG3Z3HwsKyu3LDxm9e5DS1oN\nAFw4l0YDg/SkoA8dYejH7INxE4QBAACYFEEYGKQnBX3sto4wTInZB+OmIwwAl8Dmww/k0EcOLD7m\n1ANLWg0AME8QBgatrq56ZxyeoDNPfiRXvHDx3ZPP3PXIwsd1hKEfsw/GzaXRAAAATIogDAzyjjj0\noSMM/Zh9MG4XFYSr6lhVfaKq7q6qP53te0ZV3VlVR6rqA1V1+dzxr6+qo1X16ap68dz+F1TVPVX1\nmap628WsCQAAABa52I7wY0mub6393dy+1yX5YGvt1qr6uSSvT/K6qvrOJC9P8vwkz07ywap6bmut\nJfmNJDe31u6qqvdV1Utaax+4yLUBO0RPir3g+PEHcujQRxces7n58JJWszN0hKEfsw/G7WKDcOXr\nzyq/NMn3z76+PclqtsLxDye5o7X2aJJjVXU0yXVVtZ7kaa21u2bPeWeSlyURhAHYttOnzuSKK1+0\n8Jgzj31wSasBAHazi+0ItyR/VFV3VdVPzvY9q7V2Iklaa59P8szZ/quSfG7uuffP9l2V5L65/ffN\n9gG7hHfEoQ8dYejH7INxu9gzwt/XWjteVd+c5M6qOpKtcDzv8dsAAADQzUUF4dba8dn/faCqDiW5\nLsmJqnpWa+1EVX1Lkr+dHX5/km+de/qzZ/vOt/+cbrrpplx99dVJkqc//em59tprv/KO3erqapLY\ntm17h7fPfr1b1mP70m+v/9V6vnz5l3PNP7omSXJk7UiSfM32Qw88lLPO9fi88z2+3e2HHngoR9aO\nLDz+1Be/9JWfd+KzW48/6znXfM320OPb3X70S6dz4rNHFh4/3++90PWc/R7LWO+pv//iV9Zz5Mhq\nkuSaa67/mu2hx7+yPfD/z/W/Ws/qXP+y9//ebds+1/bZfbtlPbZtj3X78OHDOXnyZJLk2LFjWZba\nulfVBTyx6h8keVJrbbOqvjHJnUn+VZIbknyhtfbW2c2yntFaO3uzrN9P8sJsXfr8R0me21prVfXx\nJD+d5K6omvpjAAAO1klEQVQk/z7Jr7XW3n+On9kudL3AhVud+4OVaThw64Gs3LCy8Jjb33x7bjxw\n46455o0/9dZc/6qfW3jMe379F/LS175xzxzzb//31+VH/7dfWspaVn/rN/MLP/PnC4+5/fZX58Yb\nf3PxMYf+m9z4r1+28Jj1D63nzf/izQuPgd7MPuijqtJaq0v9cy7mjPCzkvy7qmqz7/P7rbU7q2ot\nybur6ieSrGfrTtFprd1bVe9Ocm+SR5K8Zi7V7k/yu0m+Icn7zhWCgX78IQB96AhDP2YfjNsFB+HW\n2l8nufYc+7+Q5AfO85y3JHnLOfb/WZLvvtC1AAAAwHY9qfcCgN1vvi8FLI/PEYZ+zD4Yt4u9azQA\nMBHHT34yhz5yYPExD6zvyM86eNvBbDy4sfCYfVfuy/6b9+/IzwNgWgRhYJCeFPSx2zrCp/Nwrnjh\n4puonT7y5R35WRsPbgzesG39QzsTuuFczD4YN5dGAwAAMCmCMDBITwr60BGGfsw+GDdBGAAAgEkR\nhIFBelLQx27rCMOUmH0wboIwAAAAkyIIA4P0pKAPHWHox+yDcROEAQAAmBRBGBikJwV96AhDP2Yf\njJsgDAAAwKQIwsAgPSnoQ0cY+jH7YNwEYQAAACZFEAYG6UlBHzrC0I/ZB+MmCAMAADApl/VeALD7\nra6uemccOtiLHeHNzYdz6NBHFx5z5jObS1oNXDizD8ZNEAYAdsyZx1quuPJFC4/56y8eWtJqAODc\nXBoNDPKOOPShIwz9mH0wboIwAAAAkyIIA4N8liL0sRc7wjAWZh+MmyAMAADApLhZFjBITwr60BFe\njoO3HczGgxsLj9l35b7sv3n/klbEbmD2wbgJwgDAUh0/sZ4Dtx5YeMzaJ9aycsPKUtaz8eDG4M9a\n/9D6UtYCwHIIwsAgn6UIfYy1I3y6fXkweK5+fHU5i4HzMPtg3HSEAQAAmBRnhIFB3hGnt+PHH8ih\nQx9deMzm5sNLWs3y6AhDP2YfjJsgDMCud/rUmVxx5YsWHnPmsQ8uaTUAwF7n0mhgkM9ShD7G2hGG\nvcDsg3EThAEAAJgUQRgYpCcFfegIQz9mH4ybIAwAAMCkCMLAID0p6ENHGPox+2DcBGEAAAAmRRAG\nBulJQR86wtCP2QfjJggDAAAwKYIwMEhPCvrQEYZ+zD4Yt8t6LwCA5Tp428FsPLix8Ji1T6xl5YaV\nJa0IxmE7r619V+7L/pv3L2lFAJyPIAwM0pMal40HNwZD7urHV5ezGBYaa0d4c/PhHDr00YXHHD/+\nwJJWs3O289pa/9D6wseF6d3D7INxE4QBgKU681jLFVe+aOExp099bEd+1l67AmInwjQAwwRhYNDq\n6qp3xqEDHeGL5woILpTZB+PmZlkAAABMiiAMDPKOOPQx1o4w7AVmH4ybIAwAAMCkCMLAIJ+lCH3o\nCEM/Zh+MmyAMAADApLhrNDBITwr60BHmXNbuXsuBWw8sPMZnDV88sw/GTRAGALL58AM59JHF4Wrz\n1ANLWg2LbJ7a9FnDABdJEAYG+SxFLqXjxx/IoUMfXXjM5ubDS1rN7rLMjvCZJz+SK164OFydueuR\nJa0G+jP7YNwEYQC6On3qTK648kULjznz2AeXtBoAYArcLAsY5B1x6ENHGPox+2DcnBEGAHadzc2H\nBy+ZP/OZzSWtBoCxEYSBQXpS0MeUP0f4zGNt8JL5v/7ioSWthiky+2DcXBoNAADApDgjDAzyjjj0\noSO8e2zns3vXPrE2+LFG7B1mH4ybIAwAMGA7n927+vHV5SwGgIvm0mhg0Orqau8lwCRNuSMMvZl9\nMG6CMAAAAJPi0mhgkJ4U9KEjzKV08LaD2XhwY+Ex+67cl/0371/SinYXsw/GTRAGAPak48dP5MCB\ndyw8Zu2zn3IDq/PYeHBj8N/N+ofWl7QagOUShIFBPksR+tARXuz06ZaVlVsWHrN6t88a5sKYfTBu\nOsIAAABMijPCwCDviHOhjh9/IIcOfXThMZubDy9pNXuPjjD0Y/bBuAnCAFwyp0+dyRVXvmjhMWce\n++CSVgP04sZcwG4jCAOD9KSgDx1hxmIv3pjL7INx0xEGAABgUpwRBgZ5Rxz60BG+eMePnxjsqR8/\n/sCSVsNeYvbBuAnCAMBonT7dBnvqp099bEmrAWC3cGk0MGh1dbX3EmCSdIShH7MPxs0ZYQAA9gR3\nnwZ2iiAMDNKT4lx8RvClpyO8HJubD+sR7xHLvPv0Ts2+ofAuuEMfgjAAF8RnBNPb5sMP5NBHDiw+\n5tRwgD3zmB4xl85QeN9tHxsFUyEIA4N8liL0oSO82JknP5IrXrj47OCZux5Z0mp2l7W713Lg1sVv\nEqx9Ym3w7Ooy7bY1m30wboIwAMCAvXb59OapzcHAuPrx1eUsZpv24pq301nebW84AFsEYWCQd8T3\nDn+UjYuO8O7h8ulx2e5Nt4bm33Y6y7stvANbBGGAEfFHGfSznbPGZz6zuaTVsMgyb7q1LDt1R213\n5mYqBGFgkJ7U9Lgj9O6gI7y3bOes8V9/8dCSVrMzttPbPfqZo3nu8567+PvswStR7nz/nYPH7Kbf\na6fC/RjfJIBzEYQB+DruCA2XxvHjJ3LgwDvO+/jaZz+1a4JVsv3e7g/c8AODx+w1X3rkS66wgRET\nhIFBzgbvDvq/06MjPD6nT7esrNxy3sdX795bZ4zH7PJvvrz3EoBLSBAG2CWGgu7aJ9byoz/7owu/\nx3bOTrjsGfoZ+uzjI5/9ixw6dMXC77Gb7k4NsFcJwsAgHeHlGOpl7dQleC573jt0hMdn6LOPT931\nyODr8wsPfmhPfZTTXvXQAw/1XgJwCe2aIFxVP5jkbUmelOS21tpbOy8JmDl8+LAgvEc42zsuZ049\n0nsJ7ELbuSnXlMPydm7wtZ0ayd+f/PudXBawy+yKIFxVT0ry60luSLKR5K6qek9r7S/7rgxIkpMn\nT/ZeAtvkbO/IPNZ6r4A9asphebs3+Bry6KOP7tCKLp57RMDO2xVBOMl1SY621taTpKruSPLSJIIw\nMBlrf/ap3P3F838kxZG//Gtne4Eds1NheTv/bdrOMXsxdO+E7Z7B3ol7ROzU2XIYg90ShK9K8rm5\n7fuyFY6/zpve9KaF3+jnf/7nd25VQJLk2LFjvZfQzcGD78rGxubCY44e/XSe+9znX/wxn7k///h7\nf+q8j586/UFneyfmsUfO9F4CE7edsLyd/zZt55hlhu7tHPN3/3E5HeGdOoO9237WMm3njPm+K/dl\n/837l7SixfbaeseqWut/2VVV/WiSl7TWbplt/w9Jrmut/fTjjuu/WAAAAC6Z1lpd6p+xW84I35/k\n2+a2nz3b9zWW8S8EAACAcXtS7wXM3JXkOVW1UlVPSfKKJO/tvCYAAABGaFecEW6tnamq1ya5M1/9\n+KRPd14WAAAAI7QrOsIAAACwLEu/NLqq/ruq+mRVnamqFzzusddX1dGq+nRVvXhu/wuq6p6q+kxV\nvW1u/1Oq6o7Zcz5WVd8299iNs+OPVNWr5vZfXVUfnz32f1XVrjgrDstUVW+oqvuq6s9n//zg3GOX\n/HUIDKuqH6yqv5y9hn6u93pgr6qqY1X1iaq6u6r+dLbvGVV152w+faCqLp87fsfmIExJVd1WVSeq\n6p65fUt5rV3I35w9OsJ/keS/TfIf5ndW1fOTvDzJ85P8UJK3V9XZm2P9RpKbW2vPS/K8qnrJbP/N\nSb7QWntukrcluXX2vZ6R5BeS/JdJXpjkDXP/0t+a5Jdn3+vk7HvAFP1Ka+0Fs3/enyz1dQgsUFVP\nSvLrSV6S5LuS/PdV9R19VwV71mNJrm+t/RettbMfz/m6JB9srV2T5MNJXp8kVfWd2aE5CBP0O9ma\nW/Mu+WvtQv/mXHoQbq0daa0dTfL4O0C/NMkdrbVHW2vHkhxNcl1VfUuSp7XW7pod984kL5t7zu2z\nr/8gyT+Zff2SJHe21h5qrZ3MVvf47Bmvf5Lk386+vj1boRym6Fx3YV/W6xBY7LokR1tr6621R5Lc\nka3XGvDEVb7+b9752XV7vjrTfjgXPwdv2PHfAPaA1tofJ/m7x+2+lK+1i/qbc7fcNTpJrkryubnt\n+2f7rkpy39z++2b7vuY5rbUzSR6qqivO972q6sokf9dae2zue+3b4d8D9orXVtXhqvrtuXfNLvnr\ncMd/Cxinx79+5l9zwBPTkvxRVd1VVT852/es1tqJJGmtfT7JM2f7d2IOnpzNQSB55iV8rV3U35yX\npB9bVX+U5Fnzu7L1H6EDrbU/vBQ/c+7n7MQxsOcteh0meXuSN7bWWlW9KckvJ/nJr/8uF/ajd+j7\nAMBO+L7W2vGq+uYkd1bVkWzNw3k7efdYcxDOb9e81i5JEG6t/dMLeNr9Sb51bvvZs33n2z//nI2q\nenKSb2qtfaGq7k9y/eOe85HW2oNVdXlVPWl2Vnj+e8GoPIHX4W8lOfsG1SV/HT6R3wEm7P4k8zfc\nMa/gArXWjs/+7wNVdShb1YMTVfWs1tqJ2aWYfzs7fMfm4CX7hWBvueSvtQv9m7P3pdHzKf69SV4x\nuxvYtyd5TpI/nZ1Cf6iqrpsVqF+V5D1zz7lx9vWPZauAnSQfSPJPZ6H3GUn+6WxfsvUv5cdmX984\n971gMmb/ITrrR5J8cvb1sl6HwGJ3JXlOVa1U1VOSvCJbrzXgCaiqf1BVT519/Y1JXpytG7e+N8lN\ns8Pm/x7cyTkIU1T5+ox30+zrS/Vau6C/OZf+0UFV9bIk/0eSf5jk/66qw621H2qt3VtV705yb5JH\nkrymffVDjvcn+d0k35DkfWfvcJvktiS/V1VHkzyYrT8U0lr7u6r610nWsnX6/V/NitPJ1p3L7pg9\nfvfse8DU3FpV12brTprHkvyzJFni6xBYoLV2pqpem60bfjwpyW2ttU93XhbsRc9K8u+qqmXr797f\nb63dWVVrSd5dVT+RZD1bd6/d0TkIU1NV78rWmdkrq+pvkrwhyS8l+TeX8rV2oX9z1ld/HgAAAIxf\n70ujAQAAYKkEYQAAACZFEAYAAGBSBGEAAAAmRRAGAABgUgRhAAAAJkUQBgAAYFL+f+xQ048WTKS0\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11680c210>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "figsize(16,8)\n",
    "plt.hist(Y - lr.predict(X), range=[-1e5, 1e5], bins=100, alpha=0.4)\n",
    "plt.hist(Y - np.mean(Y), range=[-1e5, 1e5], bins=100, alpha=0.4)\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gponline, 288926.42\n",
      "seanlejgroup, 253197.89\n",
      "openwork, 174770.07\n",
      "njf, 161952.14\n",
      "maneesha, 151935.64\n",
      "medicaljobs, 148975.79\n",
      "multitie, 148917.23\n",
      "bvi, 145065.23\n",
      "theladders, 136322.30\n",
      "lhb, 135784.49\n",
      "aifmd, 132141.74\n",
      "adastra, 131189.31\n",
      "davit, 125719.57\n",
      "chanleap, 125182.56\n",
      "honquest, 124786.90\n",
      "dentistryflamehealth, 124751.26\n",
      "johnjustgpjobs, 116657.06\n",
      "noobligation, 115612.14\n",
      "workcommitment, 113934.08\n",
      "justgp, 111711.57\n",
      "hindesite, 109209.68\n",
      "brightpool, 107592.41\n",
      "jumeirah, 105723.13\n",
      "swiftsure, 102781.05\n",
      "aiv, 102066.03\n",
      "infochasemedical, 99816.96\n",
      "fragmented, 97001.80\n",
      "requiredthornbury, 96597.14\n",
      "campion, 95419.05\n",
      "...\n",
      "stakes, -59566.03\n",
      "footpath, -59871.04\n",
      "beijing, -59962.84\n",
      "mcmillan, -60596.26\n",
      "sprinting, -60941.83\n",
      "opps, -60994.21\n",
      "pres, -61060.74\n",
      "finformatiq, -61414.26\n",
      "etech, -63011.66\n",
      "cja, -63831.10\n",
      "teksystems, -64642.79\n",
      "justine, -65356.38\n",
      "inlondonis, -65788.69\n",
      "bv, -66989.12\n",
      "abbas, -67547.51\n",
      "yogurt, -67649.71\n",
      "verticality, -67724.60\n",
      "victoriacpresourcing, -68051.87\n",
      "earthquake, -69695.26\n",
      "nonexecutive, -70650.44\n",
      "convertible, -72222.51\n",
      "stipend, -72693.96\n",
      "adopter, -73096.63\n",
      "businessworks, -73551.47\n",
      "petercvscreen, -76684.95\n",
      "juniorbroker, -80238.95\n",
      "distractions, -86994.35\n",
      "johnsonaustinfraser, -93805.20\n",
      "elance, -126170.62\n"
     ]
    }
   ],
   "source": [
    "f_weights = zip(vectorizer.get_feature_names(), lr.coef_)\n",
    "f_weights = sorted(f_weights, key=lambda i: i[1])\n",
    "for i in range(1,30):\n",
    "    print '%s, %.2f' % f_weights[-i]\n",
    "    \n",
    "print '...'\n",
    "for i in reversed(range(1,30)):\n",
    "    print '%s, %.2f' % f_weights[i]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Location one-hot encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelBinarizer\n",
    "from scipy.sparse import hstack"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lb = LabelBinarizer()\n",
    "loc_features = lb.fit_transform(data.LocationNormalized.values)\n",
    "X = hstack([X, loc_features])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 26.1 s, sys: 467 ms, total: 26.6 s\n",
      "Wall time: 26.8 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "lr = Ridge(solver='sag', alpha=0.1)\n",
    "lr.fit(X,Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2 0.78\n",
      "MAE 5732.53\n",
      "RMSE 8215.62\n"
     ]
    }
   ],
   "source": [
    "print 'R2 %.2f' % r2_score(Y, lr.predict(X))\n",
    "print 'MAE %.2f' % mean_absolute_error(Y, lr.predict(X))\n",
    "print 'RMSE %.2f' % np.sqrt(mean_squared_error(Y, lr.predict(X)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# append SalaryRaw"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "all_texts = data[filter(lambda c: 'SalaryNormalized' not in c, data.columns)].values\n",
    "all_texts = map(lambda a: re.sub('[^a-z, 0-9]', ' ', ' '.join(a).lower()), all_texts.astype(str))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'chartered senior quantity surveyor award winning multi disciplinary consultancy are looking for an experienced mrics frics senior quantity surveyor to join their successful growing quantity surveying team  this is an excellent opportunity for an experienced chartered quantity surveyor to take on team management responsibilities whilst maintaining a hands on involvement in providing and growing the quantity surveying team in the south west area  you will be highly involved in business development and will work alongside the director on all cost related projects  you will be managing a small team of quantity surveyors mentoring apc junior quantity surveyors and guiding chartered qs on a variety of projects from pre contract  post contract  the successful candidate will need to be chartered mrics frics senior quantity surveyor with consultancy experience  you will need be a client facing senior quantity surveyor who has a proven track record in business development   has experience of growing a qs division  candidates must be dynamic, ambitious   enthusiastic with good interpersonal   business development skills  you will need to have a broad range of sector experience and have proven consultancy experience delivering projects in education, offices, retail,   commercial sector, some experience working with housing associations sector would be advantageous but not essential  if you feel you have the relevant experience and would like to discuss this fantastic opportunity in more detail please call lee faux   apply online only  or email somerset somerset nan permanent resourcing group consultancy jobs 45000   55000 annum cv library co uk'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_texts[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "vectorizer = TfidfVectorizer(stop_words='english', min_df=5)\n",
    "_ = vectorizer.fit(all_texts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = vectorizer.transform(all_texts)\n",
    "Y = data.SalaryNormalized.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 14.8 s, sys: 305 ms, total: 15.1 s\n",
      "Wall time: 15.4 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "lr = Ridge(solver='sag', alpha=0.1)\n",
    "lr.fit(X,Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R2 0.86\n",
      "MAE 4565.02\n",
      "RMSE 6678.40\n"
     ]
    }
   ],
   "source": [
    "print 'R2 %.2f' % r2_score(Y, lr.predict(X))\n",
    "print 'MAE %.2f' % mean_absolute_error(Y, lr.predict(X))\n",
    "print 'RMSE %.2f' % np.sqrt(mean_squared_error(Y, lr.predict(X)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seanlejgroup, 292546.56\n",
      "gponline, 277491.43\n",
      "100000, 247770.12\n",
      "120000, 208860.18\n",
      "110000, 191720.43\n",
      "100k, 191046.20\n",
      "90000, 188479.61\n",
      "350, 186627.63\n",
      "150000, 185661.13\n",
      "120k, 177799.32\n",
      "theladders, 174599.15\n",
      "80000, 173681.13\n",
      "91000, 170566.04\n",
      "85000, 166706.04\n",
      "lhb, 165098.94\n",
      "bvi, 157451.17\n",
      "99999, 155671.99\n",
      "maneesha, 155118.72\n",
      "90k, 150639.55\n",
      "95000, 149503.82\n",
      "375, 147842.93\n",
      "openwork, 147156.50\n",
      "njf, 146204.20\n",
      "75000, 142488.61\n",
      "325, 142177.64\n",
      "400, 142061.43\n",
      "380, 142057.94\n",
      "adastra, 140467.20\n",
      "aifmd, 139717.86\n",
      "...\n",
      "requiredlocation, -54060.97\n",
      "supportengineer, -54808.74\n",
      "stipend, -55606.95\n",
      "stakes, -55809.46\n",
      "currys, -56706.95\n",
      "cja, -56965.78\n",
      "managering, -58013.42\n",
      "buxtonstaractuarial, -58864.50\n",
      "muiramrgroup, -58967.59\n",
      "mta, -59864.97\n",
      "nonexecutive, -59880.52\n",
      "5500, -60172.59\n",
      "ec1v, -62230.56\n",
      "10000, -63383.25\n",
      "5000, -63749.01\n",
      "081, -64262.79\n",
      "juniorbroker, -64545.36\n",
      "jobcentre, -65088.46\n",
      "graduatefasttrack, -65900.32\n",
      "safestyle, -66735.15\n",
      "sprinting, -72795.79\n",
      "gbp0, -74811.02\n",
      "bv, -76216.99\n",
      "petercvscreen, -78757.99\n",
      "50500, -83578.08\n",
      "harrisredlawrecruitment, -85747.20\n",
      "6760, -103049.76\n",
      "elance, -105961.98\n",
      "29999, -121225.91\n"
     ]
    }
   ],
   "source": [
    "f_weights = zip(vectorizer.get_feature_names(), lr.coef_)\n",
    "f_weights = sorted(f_weights, key=lambda i: i[1])\n",
    "for i in range(1,30):\n",
    "    print '%s, %.2f' % f_weights[-i]\n",
    "    \n",
    "print '...'\n",
    "for i in reversed(range(1,30)):\n",
    "    print '%s, %.2f' % f_weights[i]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}