Updating bag-info.txt makes bag invalid

[tongwang]

When a bag has a tag manifest file created using an algorithm other than md5, such as the following bag:

bag-info.txt         
bagit.txt            
data                 
manifest-sha1.txt    
tagmanifest-sha1.txt

Updating bag info results in an invalid bag:

>>> import bagit
>>> bag = bagit.Bag('/Users/twan/edeposit/edeposit/apps/signiant/tests/SR1-4444_20140611200739')
>>> bag.is_valid()
True
>>> bag.info['External-Identifier']='xyz'
>>> bag.save()
>>> bag.is_valid()
False

The problem looks like it assumes the tag manifest file always uses md5.

# Update tag-manifest for changes to manifest & bag-info files
        _make_tagmanifest_file('tagmanifest-md5.txt', self.path)

[edsu]

@tongwang I added a test case that (I thought) would exercise the problem, but it passes. Can you get it to fail? Also, what do you see as output when you run validate()?

[tongwang]

@edsu the issue is that in the test case even you create the bag with "sha1", the tag manifest file is still in md5. It is hardwired to md5:

def _make_tagmanifest_file(tagmanifest_file, bag_dir):
    files = [f for f in listdir(bag_dir) if isfile(join(bag_dir, f))]
    checksums = []
    for f in files:
        if f == tagmanifest_file:
            continue
        fh = open(join(bag_dir, f), 'rb')
        m = hashlib.md5()
        while True:
            bytes = fh.read(16384)
            if not bytes:
                break
            m.update(bytes)
        checksums.append((m.hexdigest(), f))
        fh.close()

    with open(join(bag_dir, tagmanifest_file), 'w') as tagmanifest:
        for digest, filename in checksums:
            tagmanifest.write('%s %s\n' % (digest, filename))

[edsu]

Ouch! I thought there were tests for sha1 manifest generation. I'll check.

[edsu]

It does seem wrong that it would always write a tagmanifest-md5.txt even if you have a manifest-sha1.txt ; but why does this make the bag invalid? Can you construct a test case that fails?

[edsu]

Oh, in your example you probably have a bag created with another tool which has a tagmanifest-sha1.txt that is not being updated?

[edsu]

Ok, I've got a test case that is failing now over here.

[tongwang]

Yes, the existing tagmanifest-sha1.txt is not updated, which fails validation.

[edsu]

@tongwang can you give my latest branch a try? https://github.com/edsu/bagit/tree/issue32 I believe I fixed the issue.

[tongwang]

@edsu it works great! Thank you for the quick fix.

[edsu]

Awesome. I'll send a pull request, maybe you could merge it in (since I no longer work at LC)?