[p4dti-discussion] Error (exceptions.UnicodeDecodeError): 'ascii' codec can't decode byte 0xe4 in position 106: ordinal not in range(128)

Tue Mar 18 08:34:18 GMT 2008

Hello,

I have a simular error message like Lukas Kucera described in January 
2008.
The replicator raised exception every time we enter a perforce submit 
comment with german special characters (umlaut like ä or ß):

(P4DTI-891X)  Error (exceptions.UnicodeDecodeError): 'ascii' codec can't 
decode byte 0xe4 in position 106: ordinal not in range(128)

We have a running system with RH ES 3 (Red Hat Enterprise Linux ES release 
3 (Taroon Update 9)), 
Bugzilla 2.20.2, Perforce 2007.3 and P4DTI 2.2 and want to migrate to our 
new hardware under RH ES 5.

Please help me to solve this. Detailed info is below.
Thank you in advance.

Kind Regards
Thomas

SETUP (new server with the UnicodeDecodeError):

RH ES 5 (Red Hat Enterprise Linux Server release 5.1 (Tikanga)),
python 2.4.3
p4dti 2.4.2
bugzilla 3.0.3 (UTF8 on)
perforce 2007.3

P4DTI log:

(P4DTI-8658)  This is an automatically generated e-mail from the Perforce 
Defect
Tracking Integration replicator 'prod_repl'.

(P4DTI-8647)  The replicator failed to poll successfully, because of the
following problem:

(P4DTI-891X)  Error (exceptions.UnicodeDecodeError): 'ascii' codec can't 
decode
byte 0xe4 in position 106: ordinal not in range(128)

(P4DTI-8523)  Here's a full Python traceback:

Exception:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 106: 
ordinal not in range(128)
Traceback (innermost last):
  File "/kmtools/bugzilla/gfi/p4dti/replicator.py", line 861, in 
carefully_poll_databases
    self.poll_databases()
    locals:
      self: <replicator.replicator instance at 0xb7c7a3ac>
  File "/kmtools/bugzilla/gfi/p4dti/replicator.py", line 1220, in 
poll_databases
    self.replicate_changelist_p4_to_dt(c)
    locals:
      p4_marker: 736060
      c: {'Status': 'submitted', 'code': 'stat', 'Description': 
'Test\xe4nderung P4DTI auf EQF02\n', 'Client': 'PC08470_TST', 'User': 
'Leufkes', 'Date': '2008/03/15 09:59:26', 'Change': '102457'}
      changed_issues: <replicator.list_cursor instance at 0xb7c7aeac>
      self: <replicator.replicator instance at 0xb7c7a3ac>
      dt_marker: '2008-03-15 10:00:40'
      changed_jobs: {}
      changelists: [{'Status': 'submitted', 'code': 'stat', 'Description': 
'Test\xe4nderung P4DTI auf EQF02\n', 'Client': 'PC08470_TST', 'User': 
'Leufkes', 'Date': '2008/03/15 09:59:26', 'Change': '102457'}]
      _: {}
  File "/kmtools/bugzilla/gfi/p4dti/replicator.py", line 1268, in 
replicate_changelist_p4_to_dt
    description, status, user):
    locals:
      status: 'submitted'
      description: 'Test\xe4nderung P4DTI auf EQF02'
      changelist: {'Status': 'submitted', 'code': 'stat', 'Description': 
'Test\xe4nderung P4DTI auf EQF02\n', 'Client': 'PC08470_TST', 'User': 
'Leufkes', 'Date': '2008/03/15 09:59:26', 'Change': '102457'}
      client: 'PC08470_TST'
      user: 6
      date: '2008-03-15 09:59:26'
      self: <replicator.replicator instance at 0xb7c7a3ac>
      change: 102457
  File "/kmtools/bugzilla/gfi/p4dti/dt_bugzilla.py", line 925, in 
replicate_changelist
    self.bugzilla.add_changelist(dt_changelist)
    locals:
      status: 'submitted'
      dt_changelists: []
      description: 'Test\xe4nderung P4DTI auf EQF02'
      self: <dt_bugzilla.dt_bugzilla instance at 0xb7c7a42c>
      change: 102457
      client: 'PC08470_TST'
      user: 6
      date: '2008-03-15 09:59:26'
      dt_changelist: {'description': 'Test\xe4nderung P4DTI auf EQF02', 
'changelist': 102457, 'client': 'PC08470_TST', 'flags': 1, 'user': 6, 
'sid': 'GfI_P4Depot', 'rid': 'prod_repl', 'p4date': '2008-03-15 09:59:26'}
  File "/kmtools/bugzilla/gfi/p4dti/bugzilla.py", line 1867, in 
add_changelist
    self.insert_row_rid_sid('p4dti_changelists', dict)
    locals:
      self: <bugzilla.bugzilla instance at 0xb7c7810c>
      dict: {'description': 'Test\xe4nderung P4DTI auf EQF02', 
'changelist': 102457, 'client': 'PC08470_TST', 'flags': 1, 'user': 6, 
'sid': 'GfI_P4Depot', 'rid': 'prod_repl', 'p4date': '2008-03-15 09:59:26'}
  File "/kmtools/bugzilla/gfi/p4dti/bugzilla.py", line 615, in 
insert_row_rid_sid
    self.insert_row(table, dict)
    locals:
      table: 'p4dti_changelists'
      self: <bugzilla.bugzilla instance at 0xb7c7810c>
      dict: {'description': 'Test\xe4nderung P4DTI auf EQF02', 
'changelist': 102457, 'client': 'PC08470_TST', 'flags': 1, 'user': 6, 
'sid': 'GfI_P4Depot', 'rid': 'prod_repl', 'p4date': '2008-03-15 09:59:26'}
  File "/kmtools/bugzilla/gfi/p4dti/bugzilla.py", line 604, in insert_row
    rows = self.execute(command)
    locals:
      self: <bugzilla.bugzilla instance at 0xb7c7810c>
      value: "'2008-03-15 09:59:26'"
      command: "insert p4dti_changelists ( description, changelist, 
client, flags, user, sid, rid, p4date ) values ( 'Test\xe4nderung P4DTI 
auf EQF02', 102457, 'PC08470_TST', 1, 6, 'GfI_P4Depot', 'prod_repl', 
'2008-03-15 09:59:26' );"
      key: 'p4date'
      table: 'p4dti_changelists'
      values: "'Test\xe4nderung P4DTI auf EQF02', 102457, 'PC08470_TST', 
1, 6, 'GfI_P4Depot', 'prod_repl', '2008-03-15 09:59:26'"
      columns: 'description, changelist, client, flags, user, sid, rid, 
p4date'
      dict: {'description': 'Test\xe4nderung P4DTI auf EQF02', 
'changelist': 102457, 'client': 'PC08470_TST', 'flags': 1, 'user': 6, 
'sid': 'GfI_P4Depot', 'rid': 'prod_repl', 'p4date': '2008-03-15 09:59:26'}
  File "/kmtools/bugzilla/gfi/p4dti/bugzilla.py", line 102, in execute
    self.cursor.execute(sql)
    locals:
      self: <bugzilla.bugzilla instance at 0xb7c7810c>
      sql: "insert p4dti_changelists ( description, changelist, client, 
flags, user, sid, rid, p4date ) values ( 'Test\xe4nderung P4DTI auf 
EQF02', 102457, 'PC08470_TST', 1, 6, 'GfI_P4Depot', 'prod_repl', 
'2008-03-15 09:59:26' );"
  File "/usr/lib/python2.4/site-packages/MySQLdb/cursors.py", line 146, in 
execute
    query = query.encode(charset)
    locals:
      args: None
      self: <MySQLdb.cursors.Cursor object at 0xb7c7812c>
      charset: 'latin1'
      db: <weakproxy at 0xb7c6f464 to Connection at 0x97f697c>
      query: "insert p4dti_changelists ( description, changelist, client, 
flags, user, sid, rid, p4date ) values ( 'Test\xe4nderung P4DTI auf 
EQF02', 102457, 'PC08470_TST', 1, 6, 'GfI_P4Depot', 'prod_repl', 
'2008-03-15 09:59:26' );"
      ListType: <type 'list'>
      exc_info: <built-in function exc_info>
      TupleType: <type 'tuple'>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ravenbrook.com/pipermail/p4dti-discussion/attachments/20080318/03a2373c/attachment.html 