Ticket #40 (closed defect: fixed)

Opened 2 years ago

Last modified 2 years ago

`key in self.db` Fails when using gdbm backend

Reported by: John Hampton <pacopablo@pacopablo.com> Assigned to: athomas
Priority: major Component: pyndexter
Severity: normal Keywords: anydbm
Cc:

Description

Received the following when trying to run framework.update() 

Traceback (most recent call last):
  File "pydtest.py", line 9, in ?
    framework.update()
  File "/usr/src/pyndexter-trunk/pyndexter/__init__.py", line 727, in update
    self.index(uri)
  File "/usr/src/pyndexter-trunk/pyndexter/__init__.py", line 742, in index
    return self.indexer.index(document)
  File "/usr/src/pyndexter-trunk/pyndexter/indexers/builtin.py", line 125, in index
    old_words = self.words.get(document.uri)
  File "/usr/src/pyndexter-trunk/pyndexter/indexers/builtin.py", line 53, in get
    if key in self.db:
TypeError: iterable argument required

Don't know if it's just because I have gdbm, or if it's actually affects other databases used by anydbm. Patch attached that uses and explicit self.db.keys()

Attachments

dbkeysfix.patch (0.9 kB) - added by John Hampton <pacopablo@pacopablo.com> on 02/07/07 00:12:06.
Patch to fix traceback
keys-fix.diff (1.3 kB) - added by athomas on 02/07/07 17:33:34.
Using .keys() turns out to be excruciatingly slow. Try a key lookup and catch KeyError.

Change History

02/07/07 00:12:06 changed by John Hampton <pacopablo@pacopablo.com>

  • attachment dbkeysfix.patch added.

Patch to fix traceback

02/07/07 16:16:51 changed by John Hampton <pacopablo@pacopablo.com>

OK, after doing some big digging, this patch causes SERIOUS speed issues for update(). It is still a problem with gdbm, in that gdbm doesn't support `for key in db', but I'm not sure what to do. Proposed patch is definitely not acceptable.

While it would take around 40 seconds to index a 50K file with the patch, it takes 1 without the patch.

The amount of time it takes to index a file does increase as the number of files indexed increases, but that's a separate issue

02/07/07 17:33:34 changed by athomas

  • attachment keys-fix.diff added.

Using .keys() turns out to be excruciatingly slow. Try a key lookup and catch KeyError.

02/08/07 00:23:56 changed by athomas

  • status changed from new to closed.
  • resolution set to fixed.

Fixed in r379.

Add/Change #40 (`key in self.db` Fails when using gdbm backend)




Change Properties
Action