Today’s post has been written by Nikos Mitrakis, the creator of Forceea, an amazing Data Factory Framework for Salesforce.
Some facts about Nikos:
- Salesforce Developer at Johnson & Johnson EMEA Development Centre (EDC)
- Started his Salesforce journey in 2014
- Has passed 10 certifications, including Certified Application Architect
- Holds a Physics degree
- Leader of Limerick, Ireland Developer Group
- Married since 1994, has a daughter
- Loves watching sci-fi movies and good comedies
- Lives in Limerick, Ireland
Introduction
Dadela (pronounced dadèla) is a new programming language, designed with the intention to improve the way we generate data.
Dedala is business-oriented (not object-oriented) and it can be used by a user without programming skills (e.g. a tester or a business analyst).
The language specification is a project initiative on GitHub.
It was originally designed for Forceea data factory framework.
In Dadela we use the following language elements:
- The Environment is the system (programming language or application) that hosts Dadela’s components and capabilities, sets the execution limitations and defines the integration details for sources and destinations
- A component is a language element. There are various components: attributes, variables, lists, entity definitions, templates and operations
- A repository is a physical (e.g. a file) or a virtual (e.g. a database record) storage for the components
- A library is a special repository, containing templates and lists
- A template is a blueprint for creating records of an entity
- An entity is a database table or a matrix. It can be a Salesforce SObject, an Oracle table, an Excel worksheet, etc
- An operation is a process that creates/inserts records or exports records in text format (JSON) or CSV)
- A source is a one-way integration between an external database and the Environment for retrieving records
- A destination has the opposite direction (from the Environment to an external entity for inserting/exporting records
Lists play a key role in Dadela. A list is something like (item1, item2, item3), using a comma to separate the list items. In fact, all repository components can be considered as list items. Some examples:
number x: 100, # a numerical variable
list myList: (100, 200, 300), # a list
template BigAccounts: ( ... ), # a template
entity Account: ( ... ), # an entity definition
Lists may contain other lists or variables. Variables are referenced using @ and lists with !
number myVar1: 100,
number myVar2: 1 + myVar1, # myVar2 = 101
list myList1: (one, two, three),
string five: five, # @five = "five"
list myList2: (!myList1, four, @five), # myList2 = {one, two, three, four, five}
Of course, comments are inserted with #.
We don’t have the space to go deeper, but you’ve got the idea: the main usage of variables is to assign the same value to different components, facilitating the construction of entity definitions.
Create a template
The best way to understand a new language is “by example”. So, let’s suppose I am a business user trying to create and insert records for Opportunities, in order to prepare data for User Acceptance Testing (UAT). In my Environment I’ll create a new repository (a library), with the name “SalesTemplates” and a template in the library for the “Opportunity” entity.
Repository: SalesTemplates
template BigOpportunities: (
Name: copy field(AccountId) from(Description),
Name: static value(" - "),
Name: serial type(number) from(1) step(1) scale(0) format(000),
Amount: random type(number) from(1M) to(10M) scale(-3),
StageName: random type(picklist) except(Closed Won, Closed Lost)
)
Let’s “translate” the above, line by line:
template BigOpportunities: (
- start the definition of a template called “BigOpportunities” and execute the following steps for each generated record
Name: copy field(AccountId) from(Name),
- for the values of the Opportunity field “Name” find the Account record which is related to the lookup Opportunity field “AccountId” (the AccountId field stores the ID of a related Account record)get the value of the (Account) field “Description” from this record
Name: static value(" - "), - for the values of the Opportunity field “Name”add the string ” – ” after the previous value
Name: serial type(number) from(1) step(1) scale(0) format(000),
- for the values of the Opportunity field “Name”get serial numbers starting from 1, adding 1, with rounding to 0 decimal pointsformat the result like 001, 002, 003, …add the result after the previous value
Amount: random type(number) from(1M) to(10M) scale(-3),
- for the values of the Opportunity field “Amount” get random numbers, with minimum number 1,000,000, with maximum number 10,000,000, with rounding to 1,000
StageName: random type(picklist) except(Closed Won, losed Lost),
- for the values of the Opportunity field “StageName” get any value from the picklist values of this field, except from “Closed Won” and “Closed Lost”
)
- ends the field definitions of this template
Field definitions
Dadela has many field definitions, in 4 types (commands):
copy
- These definitions copy the value of another field or the value of a field from a related (lookup) record
static
- Static definitions just display a string, number, date/datetime or boolean value
serial
The serial definitions create serial values for
- numbers, dates, datetimes and lists
random
The random definitions create random values for
- numbers, booleans and strings, dates and datetimes, lists and picklists, lookup fields, emails, phone numbers, URLs, addresses (street, postal code, city, province, country), first and last names, and text sentences
Create the entity definition
Now, let’s continue our example. In our Environment, we create a new repository, with the name “MyEntities”.
Repository: MyEntities
language: French,
locality: France (North Europe, Europe, EU),
entity Account: (
records: 100,
alias: BigAccounts,
Rating: random type(picklist) except(Hot),
Phone: random type(phone) format("(30) 210 dD-00-DD"),
Industry: random type(picklist),
Type: random type(list) value(Prospect, Customer - Direct, Customer - Channel),
# addresses
ShippingStreet: random type(street) group(shipping),
ShippingPostalCode: random type(postalcode) group(shipping),
ShippingCity: random type(city) group(shipping),
ShippingState: random type(state) group(shipping),
ShippingCountry: random type(country) group(shipping)
),
entity Opportunity: (
template: SalesTemplates.BigOpportunities,
records: 200,
alias: MyBigOpportunities,
virtual MyField: random type(list) value(Closed Won, Closed Lost),
StageName: copy from(MyField)
)
Let’s “translate” again:
language: French,
- the default language of names (first/last) and addresses in this repository will be “French”
locality: France (North Europe, Europe, EU),
- the default geographic area (locality) in this repository will be “France”, which will be a member of other localities (“North Europe”, “Europe” and “EU”)
entity Account: ( ... ),
- set the field definitions for accounts with the default number of records to be 100 with random values for the fields Rating, Phone, Industry and Type with random “real” addresses from France, in French
entity Opportunity: ( template: SalesTemplates.BigOpportunities,
- use the field definitions from template “BigOpportunities” in repository “SalesTemplates”
virtual MyField: random type(list) value(Closed Won, Closed Lost),
- define a “virtual” field: a virtual field is like a variable
StageName: copy from(MyField)
- get the value of the virtual field (we used this for demonstration purposes – in a real situation, we use more than one virtual fields to construct a field definition)
Note that the definition for the field “Amount” will be the same as the definition in the template (this is the purpose of a template, after all).Of course, we could have defined a list for the stages inside the entity definition, or it could be a list in the repository or in a library, e.g.
entity Opportunity: (
list stages: (Closed Won, Closed Lost),
...
StageName: random type(list) value(!stages)
)
Operations
The next step is to generate and store our data.
insert
If we want to insert the records into a “database”:
- We configure a destination on our Environment. This destination could be a Salesforce org, or a SQL Server database.We use an insert operation:
insert MyBigOpportunities: (
destination(MySalesforceOrg1) group(MyGroupA)
),
The (optional) group parameter assigns a “tag” to the inserted records. This grouping is very helpful when we use the inserted records for further processing.
create
Another solution could be the create operation:
create MyBigOpportunities: (group(MyGroupA)),
Now we just generate the opportunity records and we can insert or export them later.
export
And finally, for exporting to JSON or CSV:
export MyBigOpportunities: (
destination(Ora1)
type(json) group(MyGroupA)
),
No magic here! The Environment will export our opportunities in JSON format.
Conclusion
We had a very short introduction to the syntax and capabilities of Dadela. If you are interested to learn more, you can find all the details on the GitHub project.
Closing this article, I’d like to remind that the language specifications are open for discussion and contribution to the further development and application of the language.