ChangeLog

2022-05-18    <alexander.nelson@nist.gov>

	* objects.py: Removed order-randomness in property serializations

2021-11-29    <alexander.nelson@nist.gov>

	* objects.py: Add abstract base classes for parents, children, and geometric objects

2021-08-27    <alexander.nelson@nist.gov>

	* Python: PEP 561 type designation added, though scripts and modules are still typed as needed thus far
	* objects.py: Cut copy-paste error

2021-08-20    <alexander.nelson@nist.gov>

	* Python: Reversion: `dfxml_cwd` is named `walk_to_dfxml` again
	* Python: Reversion: `dfxml_diff` is named `make_differential_dfxml` again
	* Policy: Tools are moved to `dfxml/bin/`
	* Policy: Tools installed to $PATH in setup.cfg now need documentation and unit testing

2021-08-19    <alexander.nelson@nist.gov>

	* Python: Add dfxml_diff to exported tools
	* Python: Name change - make_differential_dfxml.py is now dfxml_diff

2021-08-16    <j4n6ru@gmail.com>

	* Python: Move tools back to package installation directory tree
	* Python: Name change - walk_to_dfxml.py is now dfxml_cwd

2021-07-23    <alexander.nelson@nist.gov>

	* Python: Use setup.cfg to define installable package and command-line tools practice

2021-07-02

	* DFXML: Separate C++ and Python code bases on Github.

2020-01-03    <alexander.nelson@nist.gov>

	* Schema: Update schema pointer to 'develop'

2020-01-02    <alexander.nelson@nist.gov>

	* objects.py tests: Test partition object properties
	* objects.py: Add PartitionObject.partition_index property
	* objects.py tests: Test empty object serialization

2019-12-18    <alexander.nelson@nist.gov>

	* objects.py tests: Test storage layers with presence permutations
	* objects.py: Shore up storage layer transitions

2019-12-10    <alexander.nelson@nist.gov>

	* objects.py tests: Test error property on disk image object
	* objects.py: Add error property to disk image object
	* Schema: Track schema branch add_disk_image_error
	* objects.py tests: Reduce set of tests run by Makefile and pytest

2019-12-09    <alexander.nelson@nist.gov>

	* objects.py tests: Test adding partition object as child of partition object
	* objects.py: Add partition object as child of partition object
	* Schema: Track DFXML Schema branch add_partition_child_of_partition
	* objects.py tests: Add I/O round-trip to storage layers tests
	* objects.py: Implement "poststream" de-serialization of remaining storage layers
	* objects.py: Add parser transition for partition system containing file
	* objects.py: Add parser transition for partition system without containing disk image
	* objects.py: Whitespace-pad parser enums
	* objects.py: Confirm DiskImageObject can have FileObject 
	* objects.py tests: Confirm PartitionObject can have FileObject 
	* objects.py: Confirm DFXMLObject can have PartitionObject
	* objects.py tests: Restructure storage layers test to pytest style
	* objects.py tests: Move accessors into try-except block
	* objects.py tests: Test glomming of byte runs with hashes
	* objects.py tests: Test byte run hash serialization
	* objects.py: Add hash properties to byte runs
	* objects.py tests: Test glomming of out-of-order byte runs
	* objects.py: Correct byte run contiguity tests across independently-ordered properties
	* objects.py tests: Test byte run glomming with fill-byte
	* objects.py tests: Test glomming of contiguous byte runs
	* objects.py tests: Move byte run tests into single script call
	* objects.py: Treat hash property population with loop
	* objects.py: Correct population of _warned_elements

2019-12-02    <alexander.nelson@nist.gov>

	* objects.py tests: Add test for PartitionSystemObject error property
	* objects.py: Add error property to PartitionSystemObject, borrowing VolumeObject processing
	* Schema: Track DFXML Schema branch add_partition_system_error
	* objects.py tests: Test round-trip of external-namespace elements
	* objects.py tests: Test ways to encounter error element
	* objects.py: Revise handling of error element in parser
	* objects.py: Add poststream state to Element-stream parser
	* objects.py: Handle post-child-stream elements with independent function
	* objects.py: Handle state-based parse check with flag variable
	* objects.py: Adjust 'empty' object tests with child list checks
	* objects.py tests: Export objects.py round-trip functions to library
	* objects.py and tests: Add documenting comments
	* objects.py and tests: Adjust whitespace and variable name scheme

2019-11-04    <alexander.nelson@nist.gov>

	* Python: Relax schema presence check for local pytest builds

2019-11-01    <alexander.nelson@nist.gov>

	* Python: Remove copies of library files
	* objects.py: Add test, and new test serialization and schema-validation framework, for DiskImageObject

2019-07-05    <alexander.nelson@nist.gov>

	* setup.py: Use setuptools.find_packages
	* setup.py: Use setuptools instead of distutils
	* setup.py: Add unit test for setup.py pip deployment
	* __init__.py: Catch further commenting of _logger module-level object

2019-03-14    <alexander.nelson@nist.gov>

	* Project: Add Travis Continuous Integration
	* __init__.py: Address deprecated escape sequence warning
	* Python: Adjust unit test subprocess call to work in pytest, Pythons 2 and 3

2018-07-06    <alexander.nelson@nist.gov>

	* walk_to_dfxml.py: Add property-ignoring flag
	* Python: Add support for hashes added in DFXML 1.2.0

2018-04-09    <alexander.nelson@nist.gov>

	* Python: Update scripts' reported DFXML version to 1.2.0
	* Objects.py: Add new container classes
	* Objects.py: Rewrite iterparse loop
	* Objects.py: Normalize property listings and function recursion style
	* Objects.py: Update _warned_elements record
	* Objects.py: Adjust comment and code cosmetics

2018-02-28    <alexander.nelson@nist.gov>

	* Objects.py: Handle numeric casting and st_ino in Python 2 on Windows 7

2018-02-15    <alexander.nelson@nist.gov>

	* _pick_pythons.sh: Adjust Python selection; resolve inline comment; remove EOL'd Python3 versions
	* Python: Add disclaimer text to files created or modified at NIST.

2017-11-06    <alexander.nelson@nist.gov>

	* make_differential_dfxml.py: Revise appearance of "modified" differential annotation

2017-10-11    <alexander.nelson@nist.gov>

	* Python: Add hashdigest for SHA256 to core libraries, walk script, unit tests, and sample data.

2017-09-20    <alexander.nelson@nist.gov>

	* Python: Record runtime modules used in Objects.py-based DFXML generators.
	* Objects.py: Add LibraryObject reading and writing, and unit tests.

2017-06-18    <alexander.nelson@nist.gov>

	* walk_to_dfxml.py: Report directories.

2017-05-12    <alexander.nelson@nist.gov>

	* Python: Clean up trailing whitespace.
	* Objects.py: Add generating-program recorder and unit test.
	* Python: Record generating program in scripts using Objects.py.
	* Python: Updated DFXML generated with Objects.py to report version 1.1.1.
	* Python: Add NIST disclaimer to updated scripts.

2017-05-10    <alexander.nelson@nist.gov>

	* _pick_pythons.sh: Update list of Python 3 versions used in testing.
	* Makefile: Note phony-file targets.
	* Objects.py: Add files getter to VolumeObject.
	* Objects.py: Silence an errant warning.
	* Objects.py: Resolve XML namespace issues with DFXML printing.
	* walk_to_dfxml.py: Catch exceptions on opening files for reading.
	* Differential analysis: Add ignored file difference properties to DFXMLObject.
	* Python unit tests: Run some tests outside of in-Makefile Bash loop.
	* Python unit tests: Test for diff_file_ignores reading and writing.
	* Objects.py: Fix ElementTree name localization and Volume events in iterparse.
	* Objects.py: Add error reporting to Volume objects.
	* idifference unit test: Use variable for xmllint.
	* Python: Add NIST disclaimer to updated scripts.

2017-01-23    <alexander.nelson@nist.gov>

	* walk_to_dfxml.py: Add script to walk current directory and render as DFXML.

2015-01-21    <a.nelson@prometheuscomputing.com>

	* Python: Prevent accidental error conditions on missing a Python 3 version.

2015-01-20    <a.nelson@prometheuscomputing.com>

	* Objects.py: Fix extraction of fill bytes.

2015-01-13    <a.nelson@prometheuscomputing.com>

	* Objects.py: Add FileObject.is_allocated().

2014-10-16    <a.nelson@prometheuscomputing.com>

	* make_differential_dfxml.py: Fix configuration parameter usage.
	* Objects.py: Define RegXML properties; add 'filename' to FileObject property functions.

2014-09-29    <a.nelson@prometheuscomputing.com>

	* Unit tests: Fix test using an openssl call.

2014-09-19    <a.nelson@prometheuscomputing.com>

	* Objects.py: Allow iterparse() to run a user-specified Fiwalk build.

2014-09-16    <a.nelson@prometheuscomputing.com>

	* Objects.py: Add interface for interacting with external-namespace elements (e.g. antivirus scans per file, slackspace statistics per volume).

2014-08-21    <a.nelson@prometheuscomputing.com>

	* python: Restore Python-choosing mechanism for OS X.

2014-08-18    <a.nelson@prometheuscomputing.com>

	* Objects.py: Implement RegXML objects and differencing mechanisms.  Add unit tests.
	* python: Correct some issues with unit tests.

2014-06-04    <a.nelson@prometheuscomputing.com>

	* hash_sectors.py: Add optional tail padding.

2014-05-07    <a.nelson@prometheuscomputing.com>

	* Objects.py: Corrected buffering return rate for "fill" byte runs.

2014-04-17    <a.nelson@prometheuscomputing.com>

	* Demos: Added a new demonstration program showing construction of a MACCr timeline with the Objects interface.  Unit tests updated.
	* Objects.py: Updated documentation.

2014-03-07    <a.nelson@prometheuscomputing.com>

	* Objects.py: API change - switched "meta" byte run facet to "inode", keeping in line with "alloc_inode".  Unit tests updated.

2014-03-06    <a.nelson@prometheuscomputing.com>

	* Objects.py: Created.  New object-oriented bindings for DFXML.  Primary usage difference is better in-memory manipulation, including entire DFXML document creation without using a DFXML file; property getters and setters, instead of function calls; and a different approach to serializers and de-serializers from DFXML files.
	* make_differential_dfxml.py: Created.  Revised approach to taking differences of disk images.  Creates a differential DFXML file, or an in-memory DFXML Object if imported as a library.
	* summarize_differential_dfxml.py: Created.  Reports on disk image differences, as previously done by idifference.py.
	* idifference2.py: Created.  A re-implementation of idifference.py, using the new Object bindings, make_differential_dfxml.py, and summarize_differential_dfxml.py.
	* Differential analysis scripts: Created allocation_counter.py, break_out_diffs_by_anno.py, report_silent_changes.py
	* cat_partitions.py: Created.  Concatenates single-partition DFXML files into one DFXML file.
	* hash_sectors.py: Created.  Hashes sectors of files, storing output in a SQLite database.
	* Extractor.py: Created.  Library for general file extraction from a disk image.  A more modular iextract.py.
	* Makefile: Unit tests added.
	* test_Objects: Created.  Unit test directory for new Object bindings.
	* samples: More DFXML samples added for differencing tests.
	* Logging: Modules using the logging module now report the file that contained the call to each log message.  Previously, all calls were done with the 'root' logger.
	* dfxml_tool.py: Unit test now runs on a smaller directory tree.
	* idifference.py: Corrected a counting bug.
	* dfxml.py: Time objects can now be instantiated from floats.
	* dfxml.py: Allocation can now be parsed at a more granular level - inode and name, instead of simply "allocated."
	* dfxml.py: DFXML files with "original_fileobject" elements attached to fileobject elements can now be parsed.

2013-11-02    <a.nelson@prometheuscomputing.com>

	* idifference.py: Imported null-variable tests, and corrected a variable reference, to help idifference to be used as a module
	* idifference.py: Imported extra, granular counters on disk state changes

2013-11-01    <a.nelson@prometheuscomputing.com>

	* cat_fileobjects.py: Allow for differential DFXML repeating
	* Unit tests: Adjusted cleanup, and tested differential DFXML repeating

2013-10-24    <a.nelson@prometheuscomputing.com>

	* dfxml.py: Add null-argument test on content_for_run
	* dfxml.py: Add image reference to iterative reader
	* idifference.py: Promote delta namespace to DFXML library

2013-09-19    <a.nelson@prometheuscomputing.com>

	* idifference: Remove redundant, faulty annotation attempt

2013-09-18    <a.nelson@prometheuscomputing.com>

	* Python: Run timeline test on differencing input XML; add hand-validated line count check
	* fiwalk.py: Correct Fiwalk fallback call

2013-09-17    <a.nelson@prometheuscomputing.com>

	* dfxml: Add an ElementTree tostring() function wrapper that removes redundant namespace declarations, with unit tests for new regular expression
	* idifference: Update unit tests to include vetted count of fileobjects generated
	* idifference: Correct a fileobject counting bug (where a counter reset was forgotten)
	* idifference, icat: Add some XML namespace handling logic, found necessary but missed until the last unit test updates
	* cat_fileobjects.py: Add debug flag

2013-09-13    <a.nelson@prometheuscomputing.com>

	* idifference.py: Switch XML namespace for differencing to forensicswiki page

2013-09-12    <a.nelson@prometheuscomputing.com>

	* idifference.py: Adjust XML output, along with sample data, to validate against DFXML schema (v1.1.0rfc0; at least, everything except the differential annotations not yet in the schema validates)

2013-08-15    <a.nelson@prometheuscomputing.com>

	* Regression testing: Make demo_mac_timeline.py into regression check
	* dfxml.py: Add iterative interface for fileobjects
	* Regression testing: Add iterative version of demo_mac_timeline.py
	* Python: Add demo program for repeating fileobject with XML Python objects
	* idifference.py: Have idifference.py --xml generate DFXML with differential annotations
	* idifference.py: Add test for XML output of idifference

2013-08-13    <a.nelson@prometheuscomputing.com>

	* idifference.py: Remove requirement for a fileobject's partition to be defined

2013-06-24    <a.nelson@prometheuscomputing.com>

	* fiwalk.py: Give standalone usage slightly nicer error handling

2013-06-21    <a.nelson@prometheuscomputing.com>

	* dfxml_tool.py: Assign a version to this script (as it works pretty well now, I chose "1.0.0") - necessary for some provenance information
	* dfxml_tool.py: Optionally include provenance information (similar to Fiwalk)
	* dfxml_tool.py: Clean file-not-found error output (found on broken symlinks)
	* Python: Add 'make check' invocation for unit tests

2013-06-12    <a.nelson@prometheuscomputing.com>

	* dfxml_tool.py: Report file creation and metadata-change time, when available
	* dfxml_tool.py: Optionally output file system timestamps in ISO-8601
	* dfxml_tool.py: Optionally include directories in output listing
	* dfxml_tool.py: Include additional file metadata (partition and inode numbers)
	* dfxml_tool.py: Add alternative filename simplifier

2013-06-11    <a.nelson@prometheuscomputing.com>

	* dfxml_tool.py: Report file read failures

2013-05-28    <a.nelson@prometheuscomputing.com>

	* python: Integrated three changesets from end of 2012
	* dfxml.py, rdifference.py: Change interface of RegXML cell type
	* dfxml.py: Revise method of hashing data of Registry "value" cells
	* dfxml.py: Trim superfluous code

2013-04-23    <a.nelson@prometheuscomputing.com>

	* idifference.py: Relax input name requirements; passing a ".dfxml" file silently failed.

2013-04-08    <a.nelson@prometheuscomputing.com>

	* dfxml_tool.py (class fileobject): add filename simplifier

2012-12-18    <ajnelson@cs.ucsc.edu>

	* dfxml.py (class fileobject): correct file extraction in Python 3

2012-12-17    <ajnelson@cs.ucsc.edu>

	* dfxml.py (class dftime): add parser and tests for email timestamp format
	* dfxml.py (class fileobject): correct 'allocation' interpretation

2012-11-24    <ajnelson@cs.ucsc.edu>

	* dfxml.py (class byte_runs): change comparator methods (Python 3 doesn't use __cmp__)

2012-10-31    <simsong@ncr.nps.edu>

	* deidentify_xml.py (xml_sanitizer._start_element): fixed problem quoting attributes

2012-04-03  Simson Garfinkel  <simsong@mncrnpsedu.local>

	* dfxml.py: globally changed .data to .cdata for consistency.

2012-02-23  Simson Garfinkel  <simsong@Mucha.local>

	* dfxml.py (fileobject_reader._start_element): claned up handling of byte runs within fileobjects