Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import of data fails #208

Open
5 tasks done
sawan opened this issue Dec 3, 2018 · 0 comments
Open
5 tasks done

Import of data fails #208

sawan opened this issue Dec 3, 2018 · 0 comments

Comments

@sawan
Copy link

sawan commented Dec 3, 2018

Checklist

  • I have verified that I am using a GIS-enabled database, such as PostGIS or Spatialite.
  • I have verified that that issue exists against the master branch of django-cities.
  • I have searched for similar issues in both open and closed tickets and cannot find a duplicate.
  • I have reduced the issue to the simplest possible case.
  • I have included a failing test as a pull request. (If you are unable to do so we can still accept the issue.)

Steps to reproduce

python manage.py cities --import=all

Expected behavior

Import of all data.

Actual behavior

Fails at the index build step.

python manage.py cities --import=all
Importing countries: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 250/250 [00:00<00:00, 505.84it/s]
Importing country neighbours: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 250/250 [00:00<00:00, 389.58it/s]
Building country index: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 250/250 [00:00<00:00, 26404.51it/s]
Importing regions: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 3972/3972 [00:07<00:00, 535.45it/s]
Building region index: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 3972/3972 [00:00<00:00, 18794.72it/s]
Importing subregions: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 45731/45731 [01:35<00:00, 480.91it/s]
Building region index:   0%|                                                                                                     | 1/47993 [00:00<3:25:01,  3.90it/s]Traceback (most recent call last):
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 296, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: too many SQL variables

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/cities/management/commands/cities.py", line 160, in handle
    func()
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/cities/management/commands/cities.py", line 470, in import_city
    self.build_region_index()
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/cities/management/commands/cities.py", line 396, in build_region_index
    desc="Building region index"):
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/tqdm/_tqdm.py", line 979, in __iter__
    for obj in iterable:
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 268, in __iter__
    self._fetch_all()
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 1188, in _fetch_all
    self._prefetch_related_objects()
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 723, in _prefetch_related_objects
    prefetch_related_objects(self._result_cache, *self._prefetch_related_lookups)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 1569, in prefetch_related_objects
    obj_list, additional_lookups = prefetch_one_level(obj_list, prefetcher, lookup, level)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 1699, in prefetch_one_level
    all_related_objects = list(rel_qs)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 268, in __iter__
    self._fetch_all()
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 1186, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/query.py", line 54, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1065, in execute_sql
    cursor.execute(sql, params)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/Users/sawanvithlani/src/virtual-envs/xxx/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 296, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: too many SQL variables
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant