Related Information Examples & Tutorials

Web Host Xml Data Pump

Web Host now accepts real time XML data updates. This enables you to load, real time - into Collect!, the latest account status, notes, contacts, financials and other information provided by any external system able to send XML data.

When Web Host is started in a special mode /W3 it expects XML data for incoming data packets. The first XML tag in the incoming data packet tells Collect! the name of an import map to run when loading the data into the Collect! database.

tip.gif Web Host converts the incoming XML data into a standard comma-separated CSV file. The order of the tags in the XML data packet determines the order of the columns in the CSV file. So the import map in Collect! must be a comma-separated import map with the same layout as the fields in the XML packet.

tip.gif The intermediate CSV file that Collect! creates is placed in Collect!'s import folder. It is removed after it has been read successfully. If Web Host is started with the /whl command, the CSV files are not removed. This is very useful for troubleshooting.

How This Works

Collect! simply converts the XML data into a CSV file based on the XML tags. Then the import functionality in Collect! simply reads this CSV file and imports it based on the design of the import map. So the import map must match the layout of the XML data so that all the columns line up correctly when the CSV file is created and when it is read by Collect!'s import functions.

The Web Host shortcut properties are modified to allow direct access to the other system. This requires changing the /w1 to a /w3. This means that a login is not required and direct submits from the other system are allowed.

The external system submits the file, based on the form, to the IP address of the Web Host server.

If the client is actually using the Web Host, you would need to set up shortcuts to use different ports, one with the /w1 and one with the /w3.

1. After the HTTP header, we check for the first XML tag, for example text <noblexml>. If that is found then we set a flag isXML and invoke isXML parsing code.

2. The <noblexml>, for example, becomes the 'importmap' wrapper to select the noblexml import. Thus for this to work we need an import in Collect! named 'nobleXML'.

3. The XML tags are parsed to load each tag element into a CSV column. The name of the tag is not important but the field order is very important. It is the order of the fields that is used to create the CSV columns and these have to match the import map (in this case, 'nobleXML'.)

tip.gif The elements 'redirectto' and 'reportname' will be used if provided as in standard Web Host behavior.

redirectto Requests a specific page.

reportname Requests a report to be run after the import.

Top of page.

HTML Request Example

Submit XML File Please ...

<form name="filexml"
action="import"
enctype="multipart/form-data"
method="post">
<input type="file" name="myFile.xml">
<input type="submit" value="Upload File">
</form>

Top of page.

Example Xml Data In Myfile.Xml

<noblexml>
<listid>101</listid>
<appl>SALE</appl>
<clientaccount>1400</clientaccount>
<filler3>23434</filler3>
<call_date>10/12/2005</call_date>
<call_time>143245</call_time>
<country_id>1</country_id>
<areacode>404</areacode>
< phone>8511221</phone>
<tsr>TAC</tsr>
<status>LD</status>
<addi_status>AA</addi_status>
</noblexml>

Top of page.

Xml Data Pump Technical Notes

1. XML tags are case sensitive. </Addi_status> does NOT close <addi_status>.

2. The parser terminates on non closed fields.

3. You must have an import map named 'noblexml', for example, to import the data, i.e. an import map whose name is the same as the first tag found in the incoming file.

4. Web Host must be started with the /W3 switch to bypass authentication when using Web Host. This change makes WebUserOK() return a 1 which goes into DoHttpRequest() looking for the POST or GET command.

5. If needed, you may set a maximum packet size in Database Preferences in the Web Host Session Mem KB field.

6. Collect! does not parse packets sent as unicode or UTF-8. They must be sent in an ASCII character set, such as ISO8859-1 or one of the other sets, such as ISO8859-13, which works for Baltic countries.

Top of page.

Sample Packet

POST /import HTTP/1.1\r\n
Accept:\r\n
Content-Length: PAYLOAD_SIZE\r\n\r\n
PAYLOAD_DATA

The data packet must be text/xml, not application/xml.

tip.gif PAYLOAD_DATA does not contain a NULL terminator and is exactly PAYLOAD_SIZE bytes in length. Collect! XML handling code will be invoked by the tags <noblexml> .... </noblexml>.

Top of page.

Operator Setup

Set the Web Host up with an operator so that when you log in as that operator your Web Host can start up in Web Host mode.

Operator - Type must be "Web Host" to enable it.

Operator - Security must be "Web Host XML data pump." This will start Web Host with the /W3 switch enabled.

Operator - Detail - Just type the port number in the field.

Note 1 = /s80 - Not filling this in defaults to port 80.

Note 2 = /y443 - Not filling this in defaults to port 443.

Top of page.

Troubleshooting With Web Host Packet Logging

This shortcut argument, /whl, is used for troubleshooting transmission of Web Host data. The /whl command will also prevent the deletion of the temporary CSV files that Collect! uses for translating the XML data into CSV format for importing into Collect!.

Example:
C:\CV11\CV11.exe /S80 /W3 /y443 /whl
Web Host Packet Logging Enabled

This will generate a text file in your CV11\bin folder, i.e. "webhostpacketlog.txt." This is for advanced developer troubleshooting. This file can be used to reproduce HTTP headers and payload content, for instance in XML data pump troubleshooting.

tip.gif This will also allow you to log SSL encrypted sessions too, since Collect! writes to these files after the decryption phase.

Top of page.

Internationalization

Collect! does not parse packets sent as unicode or UTF-8. They must be sent in an ASCII character set, such as ISO8859-1 or one of the other sets, such as ISO8859-13, which works for Baltic countries.

Top of page.

See Also

- Web Host Topics

Top of page.

Was this page helpful? Do you have any comments on this document? Can we make it better? If so how may we improve this page.

Please click this link to send us your comments: helpinfo@collect.org