[Salesforce / API] Bulk API 2.0 quick & dirty!

I introduce you our week’s trailblazer Claudio Marzorati.

Claudio is a Salesforce Developer @ PwC Italy (Milan) with 2 years of experience. He worked with different retails that allowed him to increase his technical background on several aspects.
From analysis, to development, to the direct relationship with the customer, nothing is left to chance.
Other passions are running and travels!

Enjoy his quick & dirty post about Bulk API 2.0.

Since the introduction in the Winter ’18 (API version 41.0), managing bulk transactions became easier (official documentation here).

The flow to manage a job became quicker with less steps to process the set of records. To better understand the main features, here is an overview of the process for creating and managing a job with Bulk API 1.0.

If you work with few records, you won’t have many problems, but with large amount of data…it’s another story!

Working with API 1.0 means that you will follow many steps to achieve the goal.

At first you have to authenticate yourself against Salesforce to obtain a valid session ID, e.g. using SOAP APIs.

Next important thing you have to consider is that you need to pre-process your data to create N separated batches, because in this version you haven’t an automatic process that analyze and slice the data.

Here are the main manual operations:

1. Divide the data not to reach the batch size limit
2. In each batch there can’t be concurrent operations that can lock a record
3. Need to customize the header with compression or PK Chunking

You can spend a lot of time on step 2, but unlikely in Bulk API 1.0, in Bulk API 2.0 all you have to do is to upload your data and Salesforce worries about all pre-processing.

With Bulk API 2.0 you have to specify few simple steps and your job will be processed.

Below I report a simple schema with all the API 2.0 available. (The dashed component are optional and you can skip them)

I’ll provide you a quick example to start learning to use Bulk API 2.0.

We always start with a:

URL:
    https://{instance}.salesforce.com/services/oauth2/token?grant_type=password&client_id={client_id}
    &client_secret={client_secret}&username={user}&password={password+SecurityToken}

METHOD:   
    POST

To get more insights of how to get all the pieces of the call, refer to this blog post (search for OAuth Password Flow).

The response gives you the access_token (known as session id) that you need in all subsequent Bulk API 2.0 calls and the instance_url to call against:

{
  "access_token":"00D6E0000000xxx!ARIAQL7GNMpk...",
  "instance_url": "https://cs85.salesforce.com",
  "id": "https://test.salesforce.com/id/00D6E0000000XXXXXX/0056E000002LXXXXXXX",
  "token_type": "Bearer",
  "issued_at": "1520445583894",
  "signature": "rHVTsVcvZkrogMNNzxH7GpfKFlkGIHGLySS/jsVhhWc=
}

Let’s create a Job:

URL:   
   https://{instance}.salesforce.com/services/data/41.0/jobs/ingest

METHOD:   
   POST

HEADERS:   
   Content-Type: text/csv
   Authorization:Bearer {session_id}

BODY
{

  "columnDelimiter" : "COMMA",  Optional
  "externalIdFieldName": "",  Required for upsert operations
  "lineEnding": "CRLF",  Optional
  "operation": "upsert",  Required
  "object": "Account",  Required
  "contentType": "csv"     Optional
}

The response gives you back the Salesforce ID of the new batch and other configurations details:

{
   "id": "7500Y000009MxxXQAS",
   ...
}

Now put your data inside the job, paying attention to the schema/format previously defined (separator and line ending):

URL:   
   https://{instance}.salesforce.com/services/data/41.0/jobs/ingest/{jobID}/batches

METHOD:  
   PUT

HEADERS:   
   Content-Type: text/csv
   Authorization:Bearer {session_id}

BODY
   your csv input

The response has status code 201 Created if the data is compliant with the specified job configuration.

When your data is fully uploaded you can finally close your job:

URL:   
   https://{instance}.salesforce.com/services/data/41.0/jobs/ingest/{jobID}

METHOD:  
   PATCH

HEADERS:   
   Content-Type: text/csv
   Authorization:Bearer {session_id}

BODY
   {
      "state" : "UploadComplete"
   }

From now on your job will be processed, depending on your org’s queue.

If you want to check succesfull, failed or unprocessed records you can use this call:

URL:  
    https://{instance}.salesforce.com/services/async/41.0/job/{job_id}/{successfulResults|failedResults|unprocessedRecords}/

METHOD:  
   GET

HEADERS:   
   Authorization:Bearer {session_id}

The response will show the record of the csv in the requested state.

As shown Bulk API 2.0 highly simplifies the creation of bulk jobs…now it’s your turn to experiment these new APIs!