I got a question from Veronica in the comments of the Jitterbit Best Practices post asking about specifics on logging and error handling.
Out-of-the-box Jitterbit actually provides decent default logging and notifications, but the details can be really lacking. It does require a bit of tinkering to get exactly the information you want recorded or sent out exactly how you want it.
The basic strategy is to pull the information we want to see into a variable, so that we can then either use a function to write it out to log, or reference it by name in an email template.
Side note: all this info (and my previous Jitterbit post) relate to the classic, self-hosted, on-premises version only. I haven’t yet had a chance to play with the hosted “cloud” version of Jitterbit.
What to Log
This is obviously the first place to start. What extra info would we want to see in our error messages? Here are a few I’ve used:
- Jitterbit server – self explanatory, the server running the job
- Jitterbit environment (DEV/QA/UAT/PRODUCTION) – useful to quickly differentiate an error message from development from one from production.
- The operation start time
- Any server or database connection information
- The name of the operation that failed
- The error message
How to Log
For informational messages, the WriteToOperationLog() JitterScript function is your friend. This is great for recording info like start time, end time, and number of records processed in the logs so you can check them out when there hasn’t been a failure.
You can log individual records in a transformation by adding a condition to a target structure and inserting appropriate script statements there, but be aware if you have thousands of records going across, writing a log entry for each will overflow what Jitterbit can store per operation. A better option, I’ve found, is to simply log counts instead of specifics.
$company.count = $company.count + 1 $company.meetsRequirements = // some If() statement to determine if this record is included If($company.meetsRequirements, $company.includedCount = $company.includedCount + 1; true; // include , false; // don't include );
Now we have the $company.count and $company.includedCount variables to use in our logs, or in an email. A good place to record these is in a final post-execution “clean-up” script, which is very easy to do if you have a Master Operation setup (see-below).
Multiple Operations – Using a Master Operation
A related topic is how to structure multiple operations in Jitterbit. It’s very common in my experience that you’ll have a need to load several different items in sequence as part of one larger operation. It is possible to chain operations using the On Success built-in functionality of Jitterbit, so that as each finished successfully, the next is started. I find this visually confusing and prefer to use scripts to execute sub-operations of a multiple step load. We can use a simple script to execute our sub-operations:
// always prefix variables with the company name to avoid collisions with built-in Jitterbit variables $company.operationName = 'Load Widget Data'; WriteToOperationLog($company.operationName + " started."); If(!RunOperation("Load Widget Data"), // returns false if the operation run fails CancelOperationChain($company.operationName + " operation failed."); $company.fatalErrorMesage = $company.fatalErrorMesage + "\n\n" + GetLastError(); RaiseError(GetLastError()); ); WriteToOperationLog($company.operationName + " complete.");
In this way we can have a Master Operation that simply executes one or more of these scripts, and if any sub-operation fails, the information will bubble up. The Master Operation then gets assigned the failure email message, using the default Jitterbit On Failure functionality. The Master Operation also gives you a nice place to run a preparation script, which could do things like record the start date/time, as well as clean-up scripts which could handle any final tasks like recording the end date/time and writing success information out to the operation log.
Once you’ve defined the scripts that build your variables, the email templates are actually quite simple. I recommend, again, using jitterbit.conf-defined variables for the template settings, including the From and To addresses, in order to make sure you don’t have to make updates when deploying a Jitterbit package between environments. Here is an example email body:
There has been an operation failure for a Jitterbit job. Server Name : [company.jitterbit.serverName] Project Name : [company.projectName] Operation Name : [company.operationName] Operation Start Date/Time : [company.startDateTime] Source Database : [company.widgetSource.server] - [company.widgeSource.database] Target Database : [company.widgetTarget.server] - [company.widgeTarget.database] Error Message [company.fatalErrorMesage]
And here is what the subject line could look like:
[company.environmentName] - [company.projectName] Operation Failure
That’s about it. A little bit more information can make handling Jitterbit issues a lot simpler and quicker. Good luck!