notebooks/read_shapefile_from_zip.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Read a shapefile from a zip file\n",
    "\n",
    "Shapefiles are often packaged as Zip files since the spatial data is actually contained in multiple files associated with the shapefile. You don't have to unzip the zip file before loading the shapefile using Geopandas.\n",
    "\n",
    "Source: [Reading and Writing Files — GeoPandas 0.9.0 documentation](https://geopandas.org/docs/user_guide/io.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "# Local settings module\n",
    "# I use this to build paths to data files.\n",
    "# See project_settings_module.ipynb for more on this method.\n",
    "import settings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the shapefile\n",
    "\n",
    "Note that you can use the `zipinfo` command to view the contents of a zip file:\n",
    "\n",
    "```bash\n",
    "zipinfo data/src/grand_blvd.zip\n",
    "Archive:  data/src/grand_blvd.zip\n",
    "Zip file size: 4937 bytes, number of entries: 5\n",
    "drwxr-xr-x  2.1 unx        0 bx stor 19-Jul-05 12:08 grand_blvd/\n",
    "-rw-r--r--  2.1 unx      371 bX defN 19-Jul-05 12:08 grand_blvd/grand_blvd.dbf\n",
    "-rw-r--r--  2.1 unx     5404 bX defN 19-Jul-05 12:08 grand_blvd/grand_blvd.shp\n",
    "-rw-r--r--  2.1 unx      108 bX defN 19-Jul-05 12:08 grand_blvd/grand_blvd.shx\n",
    "-rw-r--r--  2.1 unx      143 bX defN 19-Jul-05 12:08 grand_blvd/grand_blvd.prj\n",
    "5 files, 6026 bytes uncompressed, 4109 bytes compressed:  31.8%\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In newer versions of GeoPandas (which rely on newer versions of [Fiona](https://github.com/Toblerity/Fiona)) you can just specify a specially-formatted path to the zip file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This file lives in `data/src/grand_blvd.zip`, relative to the root directory of this project.\n",
    "\n",
    "Within the zip file, there is a subdirectory named `grand_blvd`, which contains the shapefile `grand_blvd.shp`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "zipinfo:  cannot find or open ../data/source/grand_blvd.zip, ../data/source/grand_blvd.zip.zip or ../data/source/grand_blvd.zip.ZIP.\n"
     ]
    }
   ],
   "source": [
    "!zipinfo ../data/source/grand_blvd.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this scenario, the argument to `gpd.read_file()` would be this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'zip:///Users/GHing/workspace/python-data-cheatsheet/data/source/grand_blvd.zip!grand_blvd/grand_blvd.shp'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Path to zipped shapefile\n",
    "zipfile_path = settings.DATA_DIR_SRC / \"grand_blvd.zip\"\n",
    "# Path to shapefile within archive\n",
    "shapefile_path = Path(\"grand_blvd\") / \"grand_blvd.shp\"\n",
    "\n",
    "zipfile_url = f\"zip://{zipfile_path}!{shapefile_path}\"\n",
    "\n",
    "zipfile_url"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that because the shapefile is in a subdirectory of the zip file, there's a sub-path that appears after the zip file name in this path. The path to the archive and the shapefile within it are separated by a `!`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pri_neigh</th>\n",
       "      <th>sec_neigh</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Grand Boulevard</td>\n",
       "      <td>BRONZEVILLE</td>\n",
       "      <td>4.849250e+07</td>\n",
       "      <td>28196.837157</td>\n",
       "      <td>POLYGON ((-87.60671 41.81681, -87.60670 41.816...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         pri_neigh    sec_neigh    shape_area     shape_len  \\\n",
       "0  Grand Boulevard  BRONZEVILLE  4.849250e+07  28196.837157   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((-87.60671 41.81681, -87.60670 41.816...  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import geopandas as gpd\n",
    "from zipfile import ZipFile\n",
    "\n",
    "grand_blvd = gpd.read_file(\n",
    "    zipfile_url\n",
    ")\n",
    "\n",
    "grand_blvd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, since there's only a single shapefile in the archive, we don't have to specify the full path to the shapefile. We could just provide the path to the containing directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pri_neigh</th>\n",
       "      <th>sec_neigh</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Grand Boulevard</td>\n",
       "      <td>BRONZEVILLE</td>\n",
       "      <td>4.849250e+07</td>\n",
       "      <td>28196.837157</td>\n",
       "      <td>POLYGON ((-87.60671 41.81681, -87.60670 41.816...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         pri_neigh    sec_neigh    shape_area     shape_len  \\\n",
       "0  Grand Boulevard  BRONZEVILLE  4.849250e+07  28196.837157   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((-87.60671 41.81681, -87.60670 41.816...  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shapefile_dir_path = \"grand_blvd\"\n",
    "\n",
    "zipfile_url_dir = f\"zip://{zipfile_path}!{shapefile_dir_path}\"\n",
    "\n",
    "grand_blvd = gpd.read_file(\n",
    "    zipfile_url_dir\n",
    ")\n",
    "\n",
    "grand_blvd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Method for older versions of Geopandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With older versions of Geopandas/Fiona, you use the `vfs` keyword argument to specify the path to the zip file that contains the shapefile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/GHing/.local/share/virtualenvs/python-data-cheatsheet-ujiHmgN_/lib/python3.9/site-packages/fiona/env.py:408: FionaDeprecationWarning: The vfs keyword argument is deprecated. Instead, pass a URL that uses a zip or tar (for example) scheme.\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pri_neigh</th>\n",
       "      <th>sec_neigh</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Grand Boulevard</td>\n",
       "      <td>BRONZEVILLE</td>\n",
       "      <td>4.849250e+07</td>\n",
       "      <td>28196.837157</td>\n",
       "      <td>POLYGON ((-87.60671 41.81681, -87.60670 41.816...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         pri_neigh    sec_neigh    shape_area     shape_len  \\\n",
       "0  Grand Boulevard  BRONZEVILLE  4.849250e+07  28196.837157   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((-87.60671 41.81681, -87.60670 41.816...  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import geopandas as gpd\n",
    "from zipfile import ZipFile\n",
    "\n",
    "zipfile_url = f\"zip://{zipfile_path}\"\n",
    "grand_blvd = gpd.read_file(\n",
    "    shapefile_path,\n",
    "    vfs=zipfile_url,\n",
    ")\n",
    "\n",
    "grand_blvd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,Rmd"
  },
  "kernelspec": {
   "display_name": "python-data-cheatsheet",
   "language": "python",
   "name": "python-data-cheatsheet"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}