Loading isis/src/control/objs/ControlNetVersioner/ControlNetVersioner.h +110 −64 Original line number Diff line number Diff line Loading @@ -47,70 +47,115 @@ namespace Isis { class ControlPointV0003; /** * @brief Handle Various Control Network Versions * * This class is used to read any and all control networks. * * All publicly released versions of binary control networks should be * supported if possible. * * ControlNetVersioner::Read * / \ * [If Pvl] [If Binary] * / \ * Pvl::Read | * | [If not latest] | * Update To Latest <-- ToPvl --- ControlNetFileV????::Read * | | [If latest] * | | * Latest Pvl ------------> Latest ControlNetFile * | * | * Isis::ControlNet * * * We used to have the 4 conversions: * Pvl -> Isis::ControlNet * Binary -> Isis::ControlNet * Pvl <- Isis::ControlNet * Binary <- Isis::ControlNet * * But maintaining these causes us to need the old ControlNet code around. * These conversions are used instead: * Pvl -> Binary *Latest version only * Binary -> Pvl *All versions * Binary -> Isis::ControlNet *Latest version only * Binary <- Isis::ControlNet *Latest version only * * The log data classes are still used to understand what log data is what, * so these classes must remain backwards compatible. Otherwise all of the * versioning code is here. I encourage the use of log data to avoid needing * to make any changes in this code. * * The reason the update cycle is only in Pvl form is because of how much * simpler and less error-prone the code is to convert between versions * in a generic file format. You don't need to do things like * new.setNetworkId(old.getNetworkId()) in Pvl. Hopefully the speed cost * is not significant enough to need an update cycle in binary form. It is * a one-time cost per network, timed at about 5 minutes for our currently * largest network (120MB protocol buffer file). * * This class is the reason Isis::ControlNet only need to work with the latest * version. Also, we only need 1-way conversions for old file formats * (ControlNetFile::ToPvl). * * If you want to change the Pvl format, you must update the following: * Update LATEST_PVL_VERSION * Write ConvertVersionAToVersionB * Update ReadPvlNetwork * Update LatestPvlToBinary * * If you want to change the Binary format, you must update the following: * Update LATEST_BINARY_VERSION * Write ControlNetFileV???? * Update ControlNetFile.h * Update ReadBinaryNetwork * Update LatestPvlToBinary * @brief Handle various control network file format versions. * * This class is used to read all versions of control networks and write out * the most recent version in Pvl and protobuf format. When reading a * control net file, the ControlNeVersioner is initialized with the * filename. When writing a control net file or generating a Pvl network, * the ControlNetVersioner is initialized from a ControlNet object. * * This class exists to isolate the code dealing with control network file * formats. ControlNet can then interface with this class and only has to * work with the current ControlPoint object. * * * The read routine is as follows: * 1. Read the Pvl file header. * 2. Determine if the network is stored in Pvl or protobuf format * 3. Determine the version of the network * 4. Read in the general ControlNet information such as network * description, last modification date, etc. For Pvl networks, this * information is in a PvlObject at the start of the network. For * protobuf objects this information is stored in a header protobuf * message after the Pvl file header. * 5. For each control point do the following: * a. Read the control point into the appropriate ControlPointV#### * object. * b. If the ControlPointV#### object is not the most recent version, * upgrade it to the latest version. * c. Convert the final ControlPointV#### object into a ControlPoint * object and store it in the ControlNetVersioner. * * Once the ControlNet file is read into the ControlNetVersioner, the * ControlPoints can be accessed through the takeFirstPoint method. This * will remove the first ControlPoint stored in the ControlNetVersioner and * give it to the caller. At this point, the caller is given ownership of * the ControlPoint and is expected to delete it when finished. General * information about the control network can be accessed directly from the * ControlNetVersioner. * * * The protobuf file write routine is as follows: * 1. Copy the general ControlNet information such as network description, * last modification date, etc. into the ControlNetVersioner. * 2. Copy the pointers to the ControlPoints and store them in the * ControlNetVersioner. The ControlNetVersioner does not assume * ownership of the ControlPoints when it does this. The ControlNet * or what the ControlNet got the points from retains ownership. * 3. Write a 65536 byte blank header to the file. * 4. Write the general ControlNet information to a protobuf message header * after the blank header. * 5. For each control point do the following: * a. Convert the control point into a protobuf message. * b. Write the size of the protobuf message to the file. * c. Write the protobuf message to the file. * 6. Write a Pvl header into the original blank header at the start of the * file. This header contains a flag indicating the network is a * protobuf formatted network, the version of the format, the byte * offset and size of the protobuf header, the byte offset and * size of the block of protobuf control points, and general * information about the control network. * * * Once the ControlNetVersioner is initialized from a file or a ControlNet, * a Pvl formatted version of the control network can be created by the * toPvl method. This will always output the control network in the latest * Pvl format. From here, the Pvl network can be written to a file with * Pvl::write(filename). * * * If the control network file format is changed the following changes need * to be made to ControlNetVersioner and its supporting classes: * * New containers need to be added to interface with the new format. These * containers should try to match the format of the data in the file, then * the versioner will convert from that format to ControlNet, ControlPoint, * ControlMeasure, and ControlMeasureLogData. General information about the * control network should be stored in the ControlNetHeaderV#### structs. * Data for control points should be stored in ControlPointV#### objects. * * If a new control point container was created, code needs to be added to * create ControlPoint objects from them. A new createPoint method that * takes the new container should be added to do this. The createMeasure * method should also be changed to create ControlMeasures from the new * containers. * * If a new header container was created, code needs to be added to store its * information in the ControlNetVersioner. A new createHeader method that * takes the new header container should be added to do this. * * New code needs to be added to update the previous containers to the new * containers. Updating control point containers should happen in the new * ControlPointV#### container class. It should have a constructor that * takes a container for the previous version. Then, the createPoint method * for the previous version needs to be changed to create a new container * with this and then call the createPoint method for the new version. * Updating header containers should happen in the createHeader method that * takes the previous version. If the headers become more complicated, this * may need to change to match how control point containers are updated. * * New methods need to be added to read the new file format. Methods for * reading Pvl formatted files and protobuf formatted files need to be * added; they should match the naming convention of readPvlV#### and * readProtobufV#### respectively. * * New methods need to be added to write out the new file format. The write * method should be changed to write out the new protobuf format. If * a new header container is added, the writeHeader method should be * changed to write the new protobuf header to the file. If a new control * point container is added, the writeFirstPoint method should be changed * to write a new protobuf control point to the file. * * @ingroup ControlNetwork * Loading Loading @@ -148,6 +193,7 @@ namespace Isis { * ControlPointV#### classes. * @history 2017-12-20 Jeannie Backer - Updated toPvl and write methods to get surface point * information from the ControlPoint. * @history 2018-01-03 Jesse Mapel - Updated class documentation. * @history 2018-01-04 Adam Goins - Updated read/write methods to read/write protobuf messages * correctly. */ Loading Loading
isis/src/control/objs/ControlNetVersioner/ControlNetVersioner.h +110 −64 Original line number Diff line number Diff line Loading @@ -47,70 +47,115 @@ namespace Isis { class ControlPointV0003; /** * @brief Handle Various Control Network Versions * * This class is used to read any and all control networks. * * All publicly released versions of binary control networks should be * supported if possible. * * ControlNetVersioner::Read * / \ * [If Pvl] [If Binary] * / \ * Pvl::Read | * | [If not latest] | * Update To Latest <-- ToPvl --- ControlNetFileV????::Read * | | [If latest] * | | * Latest Pvl ------------> Latest ControlNetFile * | * | * Isis::ControlNet * * * We used to have the 4 conversions: * Pvl -> Isis::ControlNet * Binary -> Isis::ControlNet * Pvl <- Isis::ControlNet * Binary <- Isis::ControlNet * * But maintaining these causes us to need the old ControlNet code around. * These conversions are used instead: * Pvl -> Binary *Latest version only * Binary -> Pvl *All versions * Binary -> Isis::ControlNet *Latest version only * Binary <- Isis::ControlNet *Latest version only * * The log data classes are still used to understand what log data is what, * so these classes must remain backwards compatible. Otherwise all of the * versioning code is here. I encourage the use of log data to avoid needing * to make any changes in this code. * * The reason the update cycle is only in Pvl form is because of how much * simpler and less error-prone the code is to convert between versions * in a generic file format. You don't need to do things like * new.setNetworkId(old.getNetworkId()) in Pvl. Hopefully the speed cost * is not significant enough to need an update cycle in binary form. It is * a one-time cost per network, timed at about 5 minutes for our currently * largest network (120MB protocol buffer file). * * This class is the reason Isis::ControlNet only need to work with the latest * version. Also, we only need 1-way conversions for old file formats * (ControlNetFile::ToPvl). * * If you want to change the Pvl format, you must update the following: * Update LATEST_PVL_VERSION * Write ConvertVersionAToVersionB * Update ReadPvlNetwork * Update LatestPvlToBinary * * If you want to change the Binary format, you must update the following: * Update LATEST_BINARY_VERSION * Write ControlNetFileV???? * Update ControlNetFile.h * Update ReadBinaryNetwork * Update LatestPvlToBinary * @brief Handle various control network file format versions. * * This class is used to read all versions of control networks and write out * the most recent version in Pvl and protobuf format. When reading a * control net file, the ControlNeVersioner is initialized with the * filename. When writing a control net file or generating a Pvl network, * the ControlNetVersioner is initialized from a ControlNet object. * * This class exists to isolate the code dealing with control network file * formats. ControlNet can then interface with this class and only has to * work with the current ControlPoint object. * * * The read routine is as follows: * 1. Read the Pvl file header. * 2. Determine if the network is stored in Pvl or protobuf format * 3. Determine the version of the network * 4. Read in the general ControlNet information such as network * description, last modification date, etc. For Pvl networks, this * information is in a PvlObject at the start of the network. For * protobuf objects this information is stored in a header protobuf * message after the Pvl file header. * 5. For each control point do the following: * a. Read the control point into the appropriate ControlPointV#### * object. * b. If the ControlPointV#### object is not the most recent version, * upgrade it to the latest version. * c. Convert the final ControlPointV#### object into a ControlPoint * object and store it in the ControlNetVersioner. * * Once the ControlNet file is read into the ControlNetVersioner, the * ControlPoints can be accessed through the takeFirstPoint method. This * will remove the first ControlPoint stored in the ControlNetVersioner and * give it to the caller. At this point, the caller is given ownership of * the ControlPoint and is expected to delete it when finished. General * information about the control network can be accessed directly from the * ControlNetVersioner. * * * The protobuf file write routine is as follows: * 1. Copy the general ControlNet information such as network description, * last modification date, etc. into the ControlNetVersioner. * 2. Copy the pointers to the ControlPoints and store them in the * ControlNetVersioner. The ControlNetVersioner does not assume * ownership of the ControlPoints when it does this. The ControlNet * or what the ControlNet got the points from retains ownership. * 3. Write a 65536 byte blank header to the file. * 4. Write the general ControlNet information to a protobuf message header * after the blank header. * 5. For each control point do the following: * a. Convert the control point into a protobuf message. * b. Write the size of the protobuf message to the file. * c. Write the protobuf message to the file. * 6. Write a Pvl header into the original blank header at the start of the * file. This header contains a flag indicating the network is a * protobuf formatted network, the version of the format, the byte * offset and size of the protobuf header, the byte offset and * size of the block of protobuf control points, and general * information about the control network. * * * Once the ControlNetVersioner is initialized from a file or a ControlNet, * a Pvl formatted version of the control network can be created by the * toPvl method. This will always output the control network in the latest * Pvl format. From here, the Pvl network can be written to a file with * Pvl::write(filename). * * * If the control network file format is changed the following changes need * to be made to ControlNetVersioner and its supporting classes: * * New containers need to be added to interface with the new format. These * containers should try to match the format of the data in the file, then * the versioner will convert from that format to ControlNet, ControlPoint, * ControlMeasure, and ControlMeasureLogData. General information about the * control network should be stored in the ControlNetHeaderV#### structs. * Data for control points should be stored in ControlPointV#### objects. * * If a new control point container was created, code needs to be added to * create ControlPoint objects from them. A new createPoint method that * takes the new container should be added to do this. The createMeasure * method should also be changed to create ControlMeasures from the new * containers. * * If a new header container was created, code needs to be added to store its * information in the ControlNetVersioner. A new createHeader method that * takes the new header container should be added to do this. * * New code needs to be added to update the previous containers to the new * containers. Updating control point containers should happen in the new * ControlPointV#### container class. It should have a constructor that * takes a container for the previous version. Then, the createPoint method * for the previous version needs to be changed to create a new container * with this and then call the createPoint method for the new version. * Updating header containers should happen in the createHeader method that * takes the previous version. If the headers become more complicated, this * may need to change to match how control point containers are updated. * * New methods need to be added to read the new file format. Methods for * reading Pvl formatted files and protobuf formatted files need to be * added; they should match the naming convention of readPvlV#### and * readProtobufV#### respectively. * * New methods need to be added to write out the new file format. The write * method should be changed to write out the new protobuf format. If * a new header container is added, the writeHeader method should be * changed to write the new protobuf header to the file. If a new control * point container is added, the writeFirstPoint method should be changed * to write a new protobuf control point to the file. * * @ingroup ControlNetwork * Loading Loading @@ -148,6 +193,7 @@ namespace Isis { * ControlPointV#### classes. * @history 2017-12-20 Jeannie Backer - Updated toPvl and write methods to get surface point * information from the ControlPoint. * @history 2018-01-03 Jesse Mapel - Updated class documentation. * @history 2018-01-04 Adam Goins - Updated read/write methods to read/write protobuf messages * correctly. */ Loading
