Saving a notice in XML
Your user has filled in their notice on the screen and it is now time to serialise the notice data and store it in an XML file that will be submitted for publication.
| The first thing you need to worry about is getting the XML right. The XML file you generate must comply with the eForms schema. That is easier said than done, but once you get this right, you won’t have to worry about future schema changes again. Read on to see how this is done. |
The notice data is still somewhere in a data structure in your application. This is a data structure that you have designed and it probably matches the needs of your user interface. We will refer to this data structure from now on in our discussion as the visual model. See definitions.
Your goal is to get the data out of the visual model and transform it in such a way that it can easily be serialised to XML. In other words, you want to move your data to another data structure that is closer to the actual physical representation that your data will take inside the XML file. We will refer to this target data structure from now on in our discussion as the physical model. See definitions.
The problem of saving your notice data becomes therefore a transformation problem from a visual model to a physical model.
Although you can attempt to do this transformation in one go, it will probably be too complicated to get right, because the visual and physical models are just too different. One matches the end-user’s conceptualisation of a notice as it appears on their screen, while the other matches the physical structure of the notice as it is stored inside an XML file. Fields that might be grouped together on the screen, might actually be stored in totally different parts of the XML. So what you need to do to simplify things is to break this transformation into two steps:
-
Transform your notice data from your visual model to an intermediate model that breaks out of the visual structure but maintains all the information necessary to construct the physical model.
-
Transform your data from this intermediate model to the physical model.
We will refer to this intermediate model from now on in our discussion as the conceptual model.
Definitions
- Visual Model
-
A data model that the notice data adheres to while the notice is being edited by the end-user. This data model contains visual elements like notice sections, control groups and user-input controls together with the values entered by the user. We call it a visual model because its structure conforms to the structure of the notice on the end-user’s screen.
- Conceptual Model
-
A data model that follows the conceptualisation of a notice used by the eForms SDK. This data model contains
nodesandfieldsand it follows the logical structure of a notice as defined in the eForms SDK (infields/fields.json). - Physical Model
-
A data model that follows the actual physical structure that the notice data will have when stored inside the XML file. This model contains XML elements and XML attributes and follows the notice structure as defined by the eForms schema.
Conceptual model elements
node-
A
nodeis a structural element of the conceptual model. Everynodehas a parentnode(except the rootnode). Everynodehas an XPath relative to [the absolute XPath of] its parentnode. The relative XPath of anodemay contain more than onestep(each representing an XML element). XPathstepsare always separated with a forward slash/. Anodemay contain childnodesas well asfields. Anodemay or may not be repeatable. - Repeatable
node -
A repeatable
nodecan repeat (itself) inside its parentnode. When anodeis repeatable, then the XML element that repeats in the XML document, is the first XML element (the firststep) in its relative XPath. A repeatablenodemay have several other repeatablenodesin itsnode-path. node-path-
The
node-pathof anodeis a list of all the ancestors of anodein hierarchical order (starting from the rootnodeand continuing hierarchically until thenodeitself). Thenode-pathis a concept similar to the concept of XPath, where eachstepis anodeidentifier rather than an XML element. Thenode-pathof afieldis thenode-pathof its parentnodefollowed by thefieldidentifier. field-
A field is a data element in the conceptual model. Every
fieldhas a parentnode. Everyfieldhas an XPath relative to [the absolute XPath of] its parentnode. The relative XPath of afieldmay contain more than onestep(each representing an XML element or XML attribute). XPathstepsare always separated with a forward slash/. The relative XPath of afieldmay point to an XML element or an XML attribute. Afieldmay or may not be repeatable. - Repeatable
field -
A repeatable
fieldcan repeat (itself) inside its parentnode. When afieldis repeatable, then the XML element that repeats in the XML document, is the first XML element (the firststep) in its relative XPath. A repeatablefieldmay have several repeatablenodesin itsnode-path. - Multilingual
field -
A multilingual
fieldis never repeatable. However several instances of a multilingualfieldcan be created by the user to store the different text values of the field in different languages. All the instances of a multilingualfieldare meant to be created next to each other inside the parentnodeof thefield.
Visual model elements
display-group-
A
display-groupis a structural element of the visual model. It represents a section of the notice or a group ofinput-fields. Everydisplay-grouphas a parentdisplay-group(except the rootdisplay-group). Adisplay-groupmay contain several otherdisplay-groupsas well as severalinput-fields. Adisplay-groupmay or may not berepeatable. - Repeatable
display-group -
A repeatable
display-groupmay exist more than one time inside its parentdisplay-group. A repeatabledisplay-groupis always associated with a repeatablenodein the conceptual model. input-field-
An
input-fieldis a data element of the visual model. It represents an input element in the form where the user will enter data for a specificfield. Aninput-fieldis always associated with afieldin the conceptual model. Aninput-fieldmay or may not be repeatable. - Repeatable
input-field -
A repeatable
input-fieldmay exist more than one time inside its parentdisplay-group. A repeatableinput-fieldis always associated with a repeatablefieldin the conceptual model.
Step 1: Transforming from visual to conceptual model
When you constructed your visual model, you used the visual structure provided in the notice type definition files in the eForms SDK. To transform the notice data to the conceptual model you need to traverse all the elements of the visual model, one by one and follow the pseudo-algorithm presented here.
| You can traverse the visual model tree in several different ways. Choose the way that makes most sense to you. The way we chose to do it, is by executing a depth-first search in the visual model. For this discussion we assume you will use the visitor design pattern or a similar concept. |
Visiting the elements in your visual model
The first thing you need to do is create all the repeatable nodes needed in the conceptual model. This is the hardest part of the job and the most critical to get right, so take care of this first.
Here is how it works: For every repeatable node that needs to appear in the conceptual model, there is always a repeatable display-group in your visual model associated with that node. All you have to do is go though all the display-groups in your visual model and check if they are associated with a node.
You need to conduct a depth-first search starting from the root level display-groups in your visual model. To be able to do this you need to make sure that your visual model maintains its hierarchical structure. This way, you can move as needed, to the parent element and child elements of any visual element you visit.
|
If a display-group is not repeatable, continue to its first child display-group. As soon as you find a display-group that is repeatable you need to create one or more nodes for it in the conceptual model. Actually you need to create as many nodes as the number of times the end-user has repeated the display-group in the notice data.
Remember: When a node is marked as repeatable it means that the node itself repeats inside its parent node.
|
Before creating any node in the conceptual model, always make sure to create its parent node first. This way the root node of the conceptual model will always end up being created first and there will always be a parent node to which you can attach your new node. Now that the parent node exists, you can proceed with creating one node for each instance of the repeatable display-group and and attaching it to its parent node in the conceptual model.
Continue your depth-first search until you find the next repeatable display-group.
Nested repeatable display-groups
What if a repeatable display-group is nested inside one or more repeatable display-groups? When you encounter a case like that, you can safely assume that both the visual and the conceptual models always follow the same nesting pattern. For example, if you are visiting repeatable display-group-B and display-group-B has a repeatable ancestor display-group-A in the visual model, then you can assume that the repeatable node-A in the conceptual model is also an ancestor of the repeatable node-B (where node-A is the repeatable node that corresponds to the repeatable display-group-A and node-B likewise for display-group-B).
The reason why you are doing a depth-first search has just become apparent. All the ancestor repeatable display-groups have already been visited and processed, so their corresponding nodes are already created in the conceptual model. The only thing you need to do now, is to find the correct instance of the ancestor repeatable node to which to attach the new node.
To do that, you will need to use unique instance identifiers. Remember you are doing a depth-first search in the visual model and you are currently visiting a repeatable display-group. You need to find its first ancestor which is also repeatable. Get its unique identifier. Look for the node in the conceptual model with the same unique identifier. That’s the node you are looking for. From that node reconstruct the node-path until you reach the node that you are trying to attach.
Continue your depth-first search until done. Now you have a conceptual model that contains all the nodes that are needed to construct your physical model.
Visiting input-fields
When visiting an input-field you need to add a corresponding field in your conceptual model.
Much of the logic that applies when visiting an input-field is the same as the one that applies when visiting a repeatable display-group. We are keeping this description brief as we assume that you have just read the preceding part of this document and that you are already familiar with the concepts involved.
|
As when adding nodes, before adding a field in the conceptual model, always make sure that you have added its parent node first.
|
An input-field will never have a repeatable ancestor in the visual model unless that repeatable ancestor corresponds to a repeatable node that is also an ancestor of the corresponding field. Therefore, it is safe to assume that whenever you visit an input-field, that any repeatable ancestors will have already been visited and their corresponding nodes will have already been created in the conceptual model.
If the input-field has any repeatable ancestors in the visual model, then you can follow the same logic that you followed for nested repeatable display-groups: Find the first repeatable ancestor in the visual model, go to the corresponding node in the conceptual model, and from there reconstruct all the necessary nodes from the node-path of the field, until you can add the field itself.
If the input-field is repeatable, then add as many instances of the corresponding field in the conceptual model as the number of times the end-user repeated the input-field in the notice form.
If the input-field does not have any repeatable ancestors, then simply create any of the ancestor nodes that are not already present in the conceptual model. Any node that has no repeatable ancestors appears only once.
Step 2: Transforming from conceptual to physical model
You now have all the notice data in a new data structure (the conceptual model) which is much closer to the final shape that the notice data will take inside the XML file. Your goal is now to transform the notice data from the conceptual to the physical model so that you can eventually serialise them in XML.
Remember: The conceptual model contains nodes and fields. The physical model contains XML elements and attributes. Each node and field has an XPath relative to their parent node. This relative XPath contains one or more steps, separated by a forward slash '/'. Each of these steps is an XML element or XML attribute.
|
To generate your physical model you need to "unpack" the XML elements contained in the relative XPath of each of the nodes and fields in your conceptual model. To do this you need to traverse the conceptual model, element by element. Do a depth-first search again, starting from the root node.
Visiting a node
When visiting a node in the conceptual model, take its relative XPath and extract the steps it contains. Add an XML element to the physical model for each step that you extracted.
If the node is a repeatable node, then only the first step that you extracted from its relative XPath should be repeated.
|
Visiting a field
Just as you did when visiting a node, extract all the steps from the relative XPath of the field which you are currently visiting, and add them to the physical model.
If the field is repeatable, then only the XML element that corresponds to first step that you extracted should be repeated.
|
| Some fields point to XML attributes. Make sure you properly reflect that in your physical model, so that you can take it into account when serialising it to XML. |
After visiting all the nodes and fields in the conceptual model, you now have a full physical model that you can directly serialise to XML.
Getting the XML elements
One "inconvenience" that you will have to face is that you have to extract the XML elements and XML attributes that you need to add to your physical model, from the relative XPaths provided inside fields.json. Be aware that the XPaths are not always trivial as many of them contain one or more predicates.
Simply splitting the XPath string on '/' won’t work. You can use regular expressions to extract the steps. We use an XPath parser just to be on the safe side.
|
We know that extracting the steps from an XPath is not rocket science and that you can do it yourself; but it would have been much easier for everyone, if we had provided this information pre-processed and ready for you inside fields.json. This is something we intend to do in a future release of the SDK.
|
Getting the XML element order right
In fields/fields.json a property called xsdSequenceOrder is available for every field that is not pointing to an XML attribute and node that is not ND-Root.
You can read more about it in the field metadata documentation.
Simplified example field from the fields.json:
{
"id" : "BT-13713-LotResult",
"parentNodeId" : "ND-LotResult",
"xpathRelative" : "efac:TenderLot/cbc:ID",
"xsdSequenceOrder" : [ { "efac:TenderLot" : 16 }, { "cbc:ID" : 1 } ]
Here the xpathRelative has two parts, for each part we want the order of the element relative to the other child under the same parent element:
-
efac:TenderLotis at child order number 16 under the element of parent nodeND-LotResult, meaning potentially 15 elements could be in front -
cbc:IDhas child order number 1 underefac:TenderLot, meaning if it is present it must be the first child
Order of repeating elements
If a field or node is repeatable, then the repeated elements should be next to each other in the XML. The original order of the repeating items should be preserved.
Order of attributes
Even if XML is not imposing an attribute order, they could be sorted in alphabetic order. This can avoid to have variations each time the same XML is generated.
Demo sorting algorithm
In general we recommend to first build the entire notice XML and do the sorting at the end. An algorithm for sorting a notice XML is provided in the editor demo. Note that before SDK 1.8 the XML sorting algorithm in the editor demo relied only information found the schemas (XSD) of the SDK.
