Adventures in Groovy – Part 8: Customizing Data Maps and Smart Pushes
Introduction
Groovy has the ability to execute Data Maps and Smart Pushes. Data Maps are objects that define the data movement between applications and are accessible globally. Smart Pushes are Data Maps that are attached to a Data Form and can override the dimensions settings. They are largely the same thing, but defined separately inside of PBCS.
So, why execute these in Groovy rather than assign to a form and call it a day?
- The data push can be customized to push only data that has changed, improving performance.
- When a form is saved, all calculations are executed first, and the Smart pushes are executed after all calculations are finished. If Groovy is used and the data push is done inside a Business Rule, the order of operation can be a
- calculation
- data push
- calculation
- data push
- etc
- Since the Smart Push has a global limit of memory that can be used, and multiple people are running them through form saves, it is critical to make it as small as possible to reduce the probability of hitting that limit and increasing the odds of the Smart Pushes failing.
To date, I see the advantage of running a Smart Push (a Data Map on a form) in that most of the dimensions are already overridden and less overrides are required in the Groovy logic. There is no difference in performance or the size of what can be pushed between the two when executed from a Groovy calculation. The advantage of using a generic Data Map is that there is less effort in defining the form level Smart Pushes, and once one Groovy calculation is written to override all dimensions from a Data Map, it can be replicated easily to all required forms.
To understand the memory issues and explanation of how it differs from Data Maps and Smart Pushes, see PBCS Data Map / Smart Push Has Data volume Limits.
Data Map
Executing a Data Map is very simple and can be done in one line of code.
operation.application.getDataMap("Data Map Name").execute(true)
Calling execute() on a DataMap would execute the named Data Map (with no customization) and clearing the target location before pushing data. Changing the true to false, or removing it, would remove the clear and leave the data as is.
To override the the dimension settings and increase or decrease the scope of what is used, the member names need to be further defined. Every dimension’s scope can be changed, or just the ones that require a change in scope can be altered. The following changes the scope for the account, scenario, version, and Entity dimensions.
operation.application.getDataMap("Data Map Name").execute(["Account":"Net Income, Income, Expense", "Scenario":"Budget", "Version":"Working", "Entity":"East Region"], true)
Smart Push
The Smart Push works exactly the same, except the object referenced is a Smart Push, and is obtained through the grid object, not the application. Again, the likelihood that the Smart Push is further scoped is high, so it is reasonable that the dimensional parameters would be fewer as many of them change based on the POV selected.
operation.grid.getSmartPush("Smart Push Name").execute(["Account":"Min Bonus, Min Salary"])
One additional option is to define a Smart Push from a Data Map in the Groovy script.
operation.application.getDataMap("Data Map Name").createSmartPush().execute(["Account":"Min Bonus, Min Salary"])
Error Trapping
When these are written, it is likely that the Smart Pushes and Data Maps exist. One extra step to ensure the calculation doesn’t fail is to verify their existence. For Smart Pushes, verify that it is attached to the form.
//Data Map Example
if(operation.application.hasDataMap("Data Map Name"))
operation.application.getDataMap("Data Map Name").execute(true)
//Smart Push Example
if(operation.grid.hasSmartPush("Smart Push Name"))
operation.grid.getSmartPush("Smart Push Name").execute()
Conclusion
Creating variables to use in these functions to change the scope to only the rows and columns that have been edited, the calculation would look like this. This is where I see the biggest bang for your buck. To understand more about using the grid iterator, read Adventures in Groovy Part 3: Acting On Edited Cells. When a grid has hundreds of rows, only pushing the data that has been edited can make a huge difference in performance.
// Setup the variables to store the list of edited vendors and periods
def lstVendors = []
def lstPeriods = []
// Iterate through only the cells that were changed and create a list of changed vendors and periods
operation.grid.dataCellIterator({DataCell cell -> cell.edited}).each{
lstVendors.add(it.getMemberName("Vendor"))
lstPeriods.add(it.getMemberName("Period"))
}
// Convert the lists to a comma delimited string with quotes surrounding the members
String strVendors = """\"${lstVendors.unique().join('","')}\""""
String strPeriods = """\"${lstPeriods.unique().join('","')}\""""
// Execute the Smart Push with a change in scope to Vendor and Period
if(operation.grid.hasSmartPush("GP_SmartPush") && lstVendors)
operation.grid.getSmartPush("GP_SmartPush").execute(["Vendor":strVendors,"Period":strPeriods]
With any luck, you will see significant improvement, unless they change every month of every row!
Hi Kyle,
I am getting error message as “Push Data failed. Error: Not all dimensions are represented in the form” when I run SmartPush using groovy script. I am using RTP variable for 4 dimensions, Do we need to specify all the dimensions in the smartPush? I was thinking it takes from data map definition for other dimensions.
Also, can we use the Substitution variable Groovy script?
Thanks
Senthil
For subvars, see part 5. You are correct, it doesn’t required you to override all dimensions, so you can use them for 4 of them. There must be an issue with how you are passing them, unfortunately. Make sure you are sending the correct type, and make sure you aren’t sending additional single or double quotes. I have run into scenarios where ‘membername’ was invalid because I sent the actual single quotes inside the string.
Hi Kyle..
Regarding the data maps — do you know if it is it possible to submit a DM to batch from within a groovy script (similar to the feature found if the map was attached as a smart push on a form)? A few of our data maps are taking a bit longer, and I’d prefer to just get them to batch vs. the user waiting around on it.
Thank you for your hard work – love the site! Justin
To my knowledge, this isn’t possible. I am not a fan of batching it because if it fails the user doesn’t know and no issue is raised to correct. I totally understand long run times should be hidden from the user, so there is no real good answer to my argument. One option that is coming is the ability to run REST calls in a calculation. So, it might be possible to run a data map that way and not wait for it because you will be able to call it and just not monitor it.
Hi Kyle,
I’m trying to push two currencies to a reporting app (USD, which is input, and a reporting currency, which is calculated and not represented on the form). If I attach a SmartList on the form and identify both currencies in the override line things work just fine.
If I try to execute the same push and override within Groovy via a data map or by referencing the smart push on the form, the Reporting currency is left out after execution — only USD transfers. So for instance, the below line only pushes USD:
operation.grid.getSmartPush(“SmartPushName”).execute([“Currency”:”USD, USD_Reporting”])
I’ve attempted just about every creative way I can think of to force both currencies, but no luck. Any idea?
As always, thanks for the help!
Is it possible that the reporting currency is read only. Since Smartpush honors security, it has the unfortunate effect that if a member is set to RO you can’t use datapush to transfer the data. This is true even if the Datapush is wrapped in groovy rule
Oracle has on the enhancement list to add a parameter to override this and run as an admin. For now you will have to fin and alternate method. I cover a couple options in the groovy training I am offering.
Hi Kyle,
It’s usually only after I’ve declared that I’ve “tried everything” that I find a way. I pulled the data maps out of my original business logic groovy script and ran them in a separate script at the end. I’m not sure why this is the case, but it works for me. Thanks again..
It is hard to completely understand the situation you are explaining. I am glad you got it working. If you just have two pushes, I would think changing it to one and adding the currency should work. I typically create a map object to pass to these. There are less issues with spacing and such. If you don’t know that the parameter is actually a map, read the post I have on groovy methods and classes and I walk through it
def mapX = [:]
mapX[“Currency”] = “USD,USD_Reporting”
operation.grid.getSmartPush(“SmartPushName”).execute(mapX)
It makes it easier to read and override whatever you want.
Hi Kyle,
Any thoughts on how to get a basic data push to show up in the logs?
We’re getting “rule completed successfully” but no log which is worrying
Sorry for the delay. I noticed the same thing. It used to show info like the size. I could be wrong but I am al
Oat loss OROBS this is a change. I don’t know if a way to turn that back on. You could use grid builders and track the size that way, but it is work to build it just for that.
Hi Kyle,
I have an unusual requirement n that if a user changes a value in one month of the first year (a 2 year forecast), the numbers change in all subsequent months. in this cast i an creating a list of the months that changed, and I am testing to see if each month is in the lies (IE if ChangedMonths.contains(“May) pushMonths=Jun, jul, etc.) then I attempt to write out this string (pushMonths in my example) the rule validates in calc manger, but when I run it I get a “An unknown error Occured” message. I know this code is the issue since when I comment it out the process runs fine. pushMonths is a variable of type string, and the code is pushMonths=”Jun,Jul,Aug” not sure why this is happening. Any ideas you might have would be greatly appreciated.
I would strictly define your variables. My guess is that the variable is being read as a string not a list and the member with all the months doesn’t exist. It is likely reading it as a one member that is equal to Jan,feb,mar as an example, not a list of 3 members.
Hi Kyle,
I’ve used the ‘getDataMap’ syntax on a few forms that are going to be accessible by our lower level/non-admin planners. Will I have to revert to using the ‘getSmartPush’ syntax, or do you know of a trick that would allow these users to execute the datamap referenced in the groovy rule?
Thank you!!
There is no trick unfortunately. What I have done, and it is obviously different for every app, but as an example I retrieved all stored products. Then I looped through each product and use the product as an override. So you run the smart push x times. You definitely want to find the sweet spot. You want to push as much as you can each time. I have found the more I break it up the slower it gets in total. So you may want to do 10 products at a time, for example.
Thanks Kyle!!
Hello Kyle,
Thanks for you contribution to EPM community, Always been a big fan!
Here is a typical issue we are facing with customized data maps and need your expert opinion on. We need to run the map for cells updated in forms only and used groovy to extract required members and attached the rule to form to run on save. Now when a admin triggers it, it runs without any issue. But when a user triggers it, the rule doesn’t execute. We don’t see the rule in job console. We saw this is due to access issue. Now to get this data push to work, we can use the SmartPush, but due to various reason, we can’t take that option. Is there any workaround for this that you may suggest?
Thanks,
Subhasish
Sorry for the late response. If you didn’t figure this out let me know. If you can’t use smartpush due to limits, you can loop through things like dimX and run the smart push x times.
Normally I will do something like this
entity = getEval…to get the entities
entity.collate(10).each{entities->
add entities to your override map here
}