Segmentation fault processing svn repository

[eeijlar]

I get a segmentation fault while processing a SVN repository. There is no information other than:

fatal: EOF in data (285 bytes remaining)

It creates a dump report but I don't know what the contents means. It prints '+' to the screen over and over and it never appears to move past that point. It's like it has encountered a circular dependency in the svn repository.

Here's the back trace from the core dump:

[New LWP 26037]
[New LWP 26038]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `svn2git/svn-all-fast-export --identity-map ../tools/authors/authors.txt --rules'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007faaad763de1 in QByteArray::append (this=this@entry=0x189ef78, ba=...) at tools/qbytearray.cpp:1616
1616            d->data[d->size] = '\0';
(gdb) bt
#0  0x00007faaad763de1 in QByteArray::append (this=this@entry=0x189ef78, ba=...) at tools/qbytearray.cpp:1616
#1  0x0000000000418c61 in append (s=..., this=0x189ef78) at /usr/include/QtCore/qstring.h:997
#2  FastImportRepository::Transaction::addFile (this=0x189ef30, path=..., mode=33261, length=236) at repository.cpp:899
#3  0x000000000042259e in dumpBlob (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=<optimized out>, finalPathName=..., pool=pool@entry=0x7faa99e92038) at svn.cpp:292
#4  0x0000000000423fff in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xdd2218) at svn.cpp:340
#5  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xda01d8) at svn.cpp:335
#6  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xd8f1c8) at svn.cpp:335
#7  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xdc6208) at svn.cpp:335
#8  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xd6b188) at svn.cpp:335
#9  0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xdbb1f8) at svn.cpp:335
#10 0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xdb11e8) at svn.cpp:335
#11 0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=fs_root@entry=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xd861b8) at svn.cpp:335
#12 0x0000000000423f53 in recursiveDumpDir (txn=txn@entry=0x189ef30, fs_root=0xd3f6f0, pathname=..., finalPathName=..., pool=pool@entry=0xcf81f8) at svn.cpp:335
#13 0x0000000000426e07 in SvnRevision::exportInternal (this=this@entry=0x7fff44b63ca0, key=key@entry=0x189d8d8 "/FusionWorks/branches/FW_5_5_0_226", change=change@entry=0xcf9780,
    path_from=<optimized out>, path_from@entry=0xd4ee87 "/branches", rev_from=rev_from@entry=149644, current=..., rule=..., matchRules=...) at svn.cpp:914
#14 0x000000000042b56b in SvnRevision::exportDispatch (this=this@entry=0x7fff44b63ca0, key=key@entry=0x189d8d8 "/branches/FW_5_5_0_226", change=change@entry=0xcf9780,
    path_from=0xd4ee87 "/FusionWorks/branches", rev_from=149644, changes=changes@entry=0xcf96b0, current=..., rule=..., matchRules=..., pool=pool@entry=0xd4ee08) at svn.cpp:716
#15 0x000000000042cff0 in SvnRevision::exportEntry (this=this@entry=0x7fff44b63ca0, key=0x189d8d8 "/FusionWorks/branches/FW_5_5_0_226", change=0xcf9780, changes=0xcf96b0) at svn.cpp:666
#16 0x000000000042f091 in SvnRevision::prepareTransactions (this=this@entry=0x7fff44b63ca0) at svn.cpp:536
#17 0x000000000042f660 in SvnPrivate::exportRevision (this=<optimized out>, revnum=revnum@entry=149645) at svn.cpp:444
#18 0x000000000042f7ec in Svn::exportRevision (this=this@entry=0x7fff44b63e40, revnum=revnum@entry=149645) at svn.cpp:154
#19 0x0000000000407754 in main (argc=6, argv=<optimized out>) at main.cpp:276
(gdb)

[eeijlar]

Luckily the branch which caused the problem is in the stack trace, so I just did:

match /branches/broken_branch_name
end match

This allowed svn2git to skip the branch and continue on with the conversion. The errant commit seems to have propagated right through the history though. It takes about 2 hours for the process to core and before you can find the branch.

See also:
http://stackoverflow.com/questions/42828855/error-importing-svn-report-to-git-using-svn2git

It would be nice if it was able to log the branch that is causing trouble. It looks like it only logs the branch after it processes it.

[eeijlar]

In some cases you could be waiting for 6 hours for it to core dump, a better alternative is to do :

pstack <pid>

This should give you the branch it's currently processing, which you can then add to your rule set for skipping.

[eeijlar]

I think this maybe caused by a circular dependency within the svn commit history. For example if you have a project with /MyProject/branches/ and you create a branch off it /MyProject/branches/my_dev_branch/ Then if you copy the contents of /MyProject/branches/ into /MyProject/branches/my_dev_branch/

[Vampire]

We have now a test harness in place.
Could you maybe write a little test case that reproduces the problem?