index file ... is too large to fit into a mapped window ... This is a limitation of the implementation

[shuffle2]

I'm trying to use gitpython with webkit's git repo, and it fails because of this gitdb limitation.
Using gitdb from easy_install:

Downloading https://pypi.python.org/packages/source/g/gitdb/gitdb-0.5.4.tar.gz#md5=25353bb8d3ea527ba443dd88cd4e8a1c

Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from git import *
>>> repo=Repo('.')
>>> repo.branches
[<git.Head "refs/heads/master">]
>>> for commit in repo.iter_commits():
...   print commit
...   break
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\Python27x64\lib\site-packages\gitpython-0.3.2.rc1-py2.7.egg\git\repo\base.py", line 423, in iter_commits
    rev = self.head.commit
  File "c:\Python27x64\lib\site-packages\gitpython-0.3.2.rc1-py2.7.egg\git\refs\symbolic.py", line 168, in _get_commit
    obj = self._get_object()
  File "c:\Python27x64\lib\site-packages\gitpython-0.3.2.rc1-py2.7.egg\git\refs\symbolic.py", line 161, in _get_object
    return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
  File "c:\Python27x64\lib\site-packages\gitpython-0.3.2.rc1-py2.7.egg\git\objects\base.py", line 64, in new_from_sha
    oinfo = repo.odb.info(sha1)
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\db\base.py", line 256, in info
    return self._db_query(sha).info(sha)
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\db\base.py", line 239, in _db_query
    if db.has_object(sha):
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\db\pack.py", line 88, in has_object
    self._pack_info(sha)
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\db\pack.py", line 71, in _pack_info
    index = item[2](sha)
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\pack.py", line 419, in sha_to_index
    get_sha = self.sha
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\util.py", line 238, in __getattr__
    self._set_cache_(attr)
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\pack.py", line 281, in _set_cache_
    mmap = self._cursor.map()
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\util.py", line 238, in __getattr__
    self._set_cache_(attr)
  File "c:\Python27x64\lib\site-packages\gitdb-0.5.4-py2.7-win-amd64.egg\gitdb\pack.py", line 274, in _set_cache_
    raise AssertionError("The index file at %s is too large to fit into a mapped window (%i > %i). This is a limitation of the implementation" % (self
._indexpath, self._cursor.file_size(), mman.window_size()))
AssertionError: The index file at c:\src\WebKit\.git\objects\pack\pack-6a585229ec5358a9fc2e5fbd3f834ceef6902798.idx is too large to fit into a mapped
window (86149672 > 33554432). This is a limitation of the implementation

[shuffle2]

fwiw, I have 64bit OS/python and 32GB RAM, so I've just hacked it to work by changing mman in util.py:
mman = SlidingWindowMapManager(window_size = 8 * 1024 * 1024 * 1024, max_memory_size = 24 * 1024 * 1024 * 1024)

[Byron]

Thanks for the hint.
I am currently preparing a new version of smmap on pypi, which already has increased memory limits on 64bit python versions. While at it, I have upped the limits for the 32bit python interpreters as well.

This issue will be closed when smmap has been updated.

[Byron]

A new release was made on pypi