Data Quality user documentation | Clean Web Service v2

Version 2 of Clean Web Service is asynchronous and can be used to clean and validate both postal and email addresses. As such all three workflows are available.

As Version 2 is asynchronous it allows you to send a request for your records to be cleaned without having to wait for them all to be processed. Instead you send a second request to check the status of the job. This request can be sent at any interval until the job has been completed. You will then receive a result response with your cleaned and validated records.

The section below identifies the three workflows available to be sent in a request for Clean Web Service version 2.

SOAP snippet

<ns:workflow>CleanEnrich</ns:workflow>

Comments

This section must be used in every call. It informs Clean Web Service of the type of cleaning and/or validating that should take place on each Record included in the request.

Requests and responses

<soap:envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:ns="http://www.qas.com/BulkWebService/2014-10" xmlns:ns1="http://www.qas.com/BulkWebService/2011-04" xmlns:arr="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
  <soap:header xmlns:wsa="http://www.w3.org/2005/08/addressing">
    <wsa:action>http://www.qas.com/BulkWebService/2014-10/IBulkWebService/SubmitRecords</wsa:action>
    <wsa:to>https://api.experianmarketingservices.com/CleanWS/V2/BulkWebService.svc</wsa:to>
  </soap:header>
  <soap:body>
    <ns:submitrecords>
      <ns:records>
        <ns1:record>
          <ns1:fields>
            <arr:string>REF001</arr:string>
            <arr:string>MetaFiction Ltd,14 Old St,London</arr:string>
            <arr:string>contact@metafiction.com</arr:string>
          </ns1:fields>
        </ns1:record>
        <ns1:record>
          <ns1:fields>
            <arr:string>REF002</arr:string>
            <arr:string>10 Downing St,London</arr:string>
            <arr:string></arr:string>
          </ns1:fields>
        </ns1:record>
      </ns:records>
      <ns:workflow>CleanEnrichEmailValidate</ns:workflow>
      <ns:settings>
        <ns1:namedvalue>
          <ns1:key>UserLabel</ns1:key>
          <ns1:value>CleanTest</ns1:value>
        </ns1:namedvalue>
        <ns1:namedvalue>
          <ns1:key>AddressDataSet</ns1:key>
          <ns1:value>GBR</ns1:value>
        </ns1:namedvalue>
        <ns1:namedvalue>
          <ns1:key>Layout</ns1:key>
          <ns1:value>Standard</ns1:value>
        </ns1:namedvalue>
        <ns1:namedvalue>
          <ns1:key>InputFieldMappings</ns1:key>
          <ns1:value>Reference,Address,Email</ns1:value>
        </ns1:namedvalue>
        <ns1:namedvalue>
          <ns1:key>OutputHeader</ns1:key>
          <ns1:value>True</ns1:value>
        </ns1:namedvalue>
        <ns1:namedvalue>
          <ns1:key>OutputMatchProfile</ns1:key>
          <ns1:value>True</ns1:value>
        </ns1:namedvalue>
        <ns1:namedvalue>
          <ns1:key>JobLocale</ns1:key>
          <ns1:value>en-GB</ns1:value>
        </ns1:namedvalue>
      </ns:settings>
    </ns:submitrecords>
  </soap:body>
</soap:envelope>

HTTP Header

All requests sent to Clean Web Service must include an HTTP header containing your authorisation token and other parameters.

This example shows usage of the CleanEnrichEmailValidate workflow only, but changing the value of the workflow setting and omitting certain record fields and settings will make this example compatible with other workflows too.

Requests to Clean Web Service use a standard SOAP envelope which declares a bulk web service namespace (the xmlns:ns attribute). SOAP headers should be empty; authorisation tokens are sent in the HTTP header.

The contents of the request body are sent to Clean Web Service in a ns:SubmitRecords SOAP object. The following sections must be included:

ns:records

In this request, two Records are sent to Clean Web Service for processing. The Fields parameters each contain three strings, identified as string objects.

In this example, the first field of each Record is a reference field. Including a reference field makes comparisons easier, and will allow us to easily insert the processed address back into our source database. To ensure that reference fields are not processed by Clean Web Service, we map them using the InputFieldMappings setting (see below).

The next two fields of each Record are address and email fields (again, we map the order of the fields using the InputFieldMappings setting).

Any blank fields are sent as empty string objects. In this case, the second Record does not contain an email field, so this is left blank.

ns:workflow

The workflow section specifies the CleanEnrichEmailValidate workflow, meaning that the address record should be matched against postal address records and cleaned, and the email record should be cleaned and validated in real time. The results of the cleaning process can be seen in the sample response.

ns:settings

The settings section contains several NamedValues, each made up of a Key / Value pair:

The UserLabel key assigns the label 'CleanTest' to this batch of records.
The AddressDataSet key tells Clean Web Service that GBR data should be used to match records.
The Layout key tells Clean Web Service to provide the cleaned addresses using the StandardWithCoordinates layout. This is a custom layout similar to Standard, except that it includes Eastings and Northings.

A list of predefined layouts available for each country can be found here. Additional layouts may be added using the Clean Application user interface, or by contacting your Experian Data Quality Account Manager.

The InputFieldMappings key tells Clean Web Service that the first field in each row is for reference purposes only, the second field contains address data and the third field contains an email address. The order of components in this setting must correspond to the order of the fields in each Record block.
The OutputHeader key tells Clean to include an additional Record block at the beginning of the response (before the first cleaned record) which contains headers for each field in all subsequent Record blocks.
The OutputMatchProfile key tells Clean to include an additional string in each Record block which details the Match Profile for the address. For more information, see Match Profiles.
The JobLocale key tells the Clean which language the results should be sent in. Only OutputHeader and OutputMatchProfile are affected by this setting. Supported values are en-US, en-GB or fr-FR.

Each of these keys are covered in more detail in the SOAP Method Reference section.

<s:envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
  <s:header>
    <a:action s:mustunderstand="1">http://www.qas.com/BulkWebService/2011-04/IBulkWebService/ProcessRecordsResponse</a:action>
    <a:relatesto>uuid:a8bdb736-3663-4a72-98d4-104226bc5f13</a:relatesto>
  </s:header>
  <s:body>
    <submitrecordsresponse xmlns="http://www.qas.com/BulkWebService/2014-10">
      <submitrecordsresult xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
        <additionalinformation xmlns:b="http://www.qas.com/BulkWebService/2011-04">
          <jobid>1225f4e3-f20f-4766-8334-9ad9b923aa41</jobid>
          <message>Job in progress.</message>
          <records>
            xmlns:b="http://www.qas.com/BulkWebService/2011-04"/>
            <status>Processing</status>
          </records></additionalinformation></submitrecordsresult>
    </submitrecordsresponse>
  </s:body>
</s:envelope>

Responses are sent from Clean Web Service in a SubmitRecordsResult SOAP object. The following sections are included:

RelatesTo

This section reflects an ID that was passed by the caller in the header of the request that generated the response.

JobId

A unique JobId is included in every response. If you need to contact Experian regarding one of your jobs, this can be used for tracing purposes.

Message

The Message field contains general information about the job, including any errors that may have been encountered. In this example, we can see that the job is still in progress.

Status

The Status field contains a short summary of the job. In this example, this tells us that our request is processing.

<soap:envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"></soap:envelope><soap:header xmlns:wsa="http://www.w3.org/2005/08/addressing">
    <wsa:action>http://www.qas.com/BulkWebService/2014-10/IBulkWebService/GetResults</wsa:action>
    <wsa:to>https://api.experianmarketingservices.com/CleanWS/V2/BulkWebService.svc</wsa:to>
  </soap:header>
  <soap:body>
    <ns:getresults>
      <ns:jobid>1225f4e3-f20f-4766-8334-9ad9b923aa41</ns:jobid>
    </ns:getresults>
  </soap:body>

HTTP Header

All requests sent to Clean Web Service must include an HTTP header containing your authorisation token and other parameters.

The contents of the request body are sent to Clean Web Service in a ns:GetResults SOAP object. The only section to be included is:

JobId

The JobId is the only section for this request and is required in order to get the results of the job we submitted. The JobId can be found in the relevant SubmitRecordsResponse. If you need to contact Experian regarding one of your jobs, this can be used for tracing purposes.

Version 2 of Clean Web Service is asynchronous and so therefore when sending the GetResults request there are two possible responses.

Response 1

If the results are not ready, you will get a response similar to the one below to explain that the job is still in progress.

<s:envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
  <s:header>
    <a:action s:mustunderstand="1">http://www.qas.com/BulkWebService/2011-04/IBulkWebService/ProcessRecordsResponse</a:action>
    <a:relatesto>uuid:a8bdb736-3663-4a72-98d4-104226bc5f13</a:relatesto>
  </s:header>
  <s:body>
    <getresultsresponse xmlns="http://www.qas.com/BulkWebService/2014-10">
      <getresultsresult xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
        <additionalinformation xmlns:b="http://www.qas.com/BulkWebService/2011-04">
          <b:namedvalue>
            <b:key>ProcessingTime</b:key>
            <b:value>32141.61ms</b:value>
            <jobid>1225f4e3-f20f-4766-8334-9ad9b923aa41</jobid>
            <message>Job in progress.</message>
            <records>
              xmlns:b="http://www.qas.com/BulkWebService/2011-04"/>
              <status>Processing</status>
            </records></b:namedvalue></additionalinformation></getresultsresult>
    </getresultsresponse>
  </s:body>
</s:envelope>

Response 2

If the job has completed then you will receive a response similar to the one below.

<s:envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
  <s:header>
    <a:action s:mustunderstand="1">http://www.qas.com/BulkWebService/2014-10/IBulkWebService/GetResultsResponse</a:action>
  </s:header>
  <s:body>
    <getresultsresponse xmlns="http://www.qas.com/BulkWebService/2014-10">
      <getresultsresult xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
        <additionalinformation xmlns:b="http://www.qas.com/BulkWebService/2011-04">
          <namedvalue>
            <key>TotalRecords</key>
            <value>2</value>
          </namedvalue>
          <namedvalue>
            <key>RecordsProcessed</key>
            <value>2</value>
          </namedvalue>
          <namedvalue>
            <key>RecordsSubmitted</key>
            <value>2</value>
          </namedvalue>
          <namedvalue>
            <key>ProcessingTime</key>
            <value>8328.3915ms</value>
          </namedvalue>
          <namedvalue>
            <key>BatchVersion</key>
            <value>7.30</value>
          </namedvalue>
          <namedvalue>
            <key>DatasetName</key>
            <value>GBR</value>
          </namedvalue>
          <namedvalue>
            <key>DataSetVintage</key>
            <value>21/02/2014</value>
          </namedvalue>
          <namedvalue>
            <key>CLEAN-GBR_RecordsCleaned</key>
            <value>2</value>
          </namedvalue>
          <namedvalue>
            <key>CLEAN-GBR_RecordsProcessed</key>
            <value>2</value>
          </namedvalue>
          <namedvalue>
            <key>CLEAN-GBR-GBRGRD_Dataplus_RecordsCleaned</key>
            <value>2</value>
          </namedvalue>
          <namedvalue>
            <key>CLEAN-GBR-GBRGRD_Dataplus_RecordsProcessed</key>
            <value>2</value>
          </namedvalue>
          <namedvalue>
            <key>CLEAN-EMAILVALIDATE_RecordsProcessed</key>
            <value>1</value>
          </namedvalue>
        </additionalinformation>
        <jobid>1225f4e3-f20f-4766-8334-9ad9b923aa41</jobid>
        <message>Records processed successfully.</message>
        <records>
          <record>
            <fields xmlns:b="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
              <b:string>MatchProfile</b:string>
              <b:string>MatchCode</b:string>
              <b:string>ReferenceField</b:string>
              <b:string>CleanedAddress1</b:string>
              <b:string>CleanedAddress2</b:string>
              <b:string>CleanedAddress3</b:string>
              <b:string>CleanedAddress4</b:string>
              <b:string>CleanedAddress5</b:string>
              <b:string>CleanedAddress6</b:string>
              <b:string>CleanedAddress7</b:string>
              <b:string>CleanedEmail</b:string>
              <b:string>EmailMatchCode</b:string>
            </fields>
          </record>
          <record>
            <fields xmlns:b="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
              <b:string>Good Premise Partial</b:string>
              <b:string>P923004020220000100000000000:GBR</b:string>
              <b:string>REF001</b:string>
              <b:string>14-18 Old Street</b:string>
              <b:string></b:string>
              <b:string>LONDON</b:string>
              <b:string></b:string>
              <b:string>EC1V 9BH</b:string>
              <b:string>05320</b:string>
              <b:string>01822</b:string>
              <b:string>contact@metafiction.com</b:string>
              <b:string>Verified</b:string>
            </fields>
          </record>
          <record>
            <fields xmlns:b="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
              <b:string>Good Premise Partial</b:string>
              <b:string>P923000000220000000000000000:GBR</b:string>
              <b:string>REF002</b:string>
              <b:string>10 Downing Street</b:string>
              <b:string></b:string>
              <b:string>LONDON</b:string>
              <b:string></b:string>
              <b:string>SW1A 2AA</b:string>
              <b:string>05300</b:string>
              <b:string>01799</b:string>
              <b:string></b:string>
              <b:string>Unreachable</b:string>
            </fields>
          </record>
        </records>
        <status>Completed</status>
      </getresultsresult>
    </getresultsresponse>
  </s:body>
</s:envelope>

Responses are sent from Clean Web Service in a GetResultsResult SOAP object. The following sections are included:

This example shows a response for the CleanEnrichEmailValidate workflow only; responses from other workflows are similar, but omit values specific to postal or email addresses.

RelatesTo

This section reflects an ID that was passed by the caller in the header of the request that generated the response. If this was not present in the request then this section will not appear in the response.

AdditionalInformation

The first section of each response is AdditionalInformation, which contains information regarding the data used to process and clean the records. This information is contained within NamedValues, which are in turn made up of Key / Value pairs:

the DataSetVintage key details the age of the data which was used to match address records. Results of address cleaning may vary slightly depending on the vintage of data that is used, as newer vintages contain updated data. Clean Web Service always uses the latest data as soon as it is available.
the […]RecordsProcessed keys detail how many address records Clean Web Service was able to match against each cleaning process or postal address dataset. In the example above, we can see that our postal address records were successfully processed against the core GBR dataset (CLEAN-GBR) and the GBR grid reference dataset (CLEAN-GBR-GBRGRD). This corresponds to the AddressDataset and Layout settings sent as part of the request. We can also see that one email address was successfully validated (CLEAN-EMAILVALIDATE). This is as expected as we only provided one email address.

If any records could not be cleaned successfully against the GBR dataset, Clean Web Service would not attempt to enrich the record by appending GBR-GBRGRD DataPlus information to it. In this situation, neither the RecordsProcessed or RecordsCleaned values for any datasets would be incremented.

the […]RecordsCleaned keys detail how many records were successfully cleaned using each postal address dataset. In the example above, we can see that our postal address records were cleaned using both datasets which were used to process them (the GBR dataset, explicitly specified in the request using the AddressDataset setting, and the GBR-GBRGRD dataset and some of its DataPlus sets, implicitly included by the Layout *setting *which was specified in the request). If any records could not be cleaned successfully against the GBR dataset, Clean Web Service would not attempt to enrich the record by appending GBR-GBRGRD DataPlus information to it. In this situation, neither the RecordsProcessed or RecordsCleaned values for any datasets would be incremented.

For more information, see this definition of postal address processing and cleaning.

The TotalRecords, RecordsProcessed and RecordsSubmitted keys details the total number of records in the response, the number of records that were successfully processed and the number of records that were originally sent in the corresponding request.
The ProcessingTime key details the time Clean took to process the request, in milliseconds.

JobId

A unique JobId is included in every response. If you need to contact Experian regarding one of your jobs, this can be used for tracing purposes.

Message

The Message field contains general information about the completed job, including any errors that may have been encountered. In the first example, we can see that the job is still in progress. In the second example, we can see all records were processed successfully.

Records

Each completed job response contains cleaned Records – one for each Record you sent in a SOAP request. Each Record's Fields are represented as string objects.

If you set the OutputHeader setting to True, the first Record will contain headers for each field in all subsequent Record blocks (for example, 'Match Profile', 'ReferenceField', 'CleanedAddress1', 'CleanedEmail').

In all subsequent Record blocks in this example:

The first string is the postal address match profile (if this was requested using the OutputMatchProfile setting). Match profiles define Clean's level of success in terms of cleaning this specific postal address Record. Match profiles are closely related to match codes.
The next string is always the postal address match code. In the example above, each match code begins with P or R and ends with GBR, which tells us that Clean Web Service successfully matched the postal address records to addresses in the United Kingdom.
In this example, the next string is the reference field. If used, this is always the same as the reference field you supplied.
The next five strings (in this example) are separate address lines, organised according to the Layout setting specified in the request.
The next two strings contain two DataPlus elements, which are implicitly specified by the Layout setting of the request (in this case, this is the Easting and Northing coordinates of the addresses).
The next string is the email address. If no email address is supplied in the request, or if the email address has been found to be invalid, this string will be blank.
The string after the email address is the email match code. Email match codes define Clean's level of success in terms of cleaning this specific email address.

Status

The Status field contains a short summary of the completed job. In this example, this tells us that our request was completed without errors or timeouts.

Was this helpful?

Previous: Clean Web Service v1

Next: Match profiles

Clean Web Service

API reference

Previous topic:
Pre-requisites