Adventures in Groovy – Part 12: Learning and Testing Groovy Outside of PBCS

Introduction

For people that are new to Groovy/Java, testing functions that Groovy provides can be a tedious and time consuming process.  Learning anything is.  Trying to do this with the wrong tools compounds it.  I have seen some people give up and walk away from trying to improve applications because they struggle with the Groovy Calculations and the complexity it introduces to go beyond some of the basics, just because they are using a hammer when they need a screwdriver.  For example, it is simple to use a documented example and loop through the cells on a form, but to utilize the Groovy/Java objects and methods is the difference between using the default logic and taking Planning to a whole new level.  For those of us who are learning, testing simple functions can be very painful inside a Groovy Calculation.

I will by preface saying I am not a Groovy developer.  I am learning as I need functionality and I am trying to build a foundation to be as productive as possible.  Although Groovy in PBCS doesn’t give developers full access to all the Java libraries, much of the logic that is needed to develop new functionality can be tested outside of PBCS.  I have found that as I learn more and require more non PBCS related functionality, it is easier to test in the Groovy Console rather than in a PBCS calculation.  Some examples are

  • string functions like replace, regex, concatenate
  • mathematical functions
  • other manipulation that require the use of collections and hash tables

These can be used in looping through grid cells or building evaluation rules on data entered.  Hopefully, this is helpful to those learning Groovy.

How To Get Started

Download Java SDK

Before Groovy can be used, Java has to be installed.  Most systems already have it.  If not, the Java Development Kit can be downloaded and installed.  There is information about which version of Groovy and Java are compatible at groovy-lang.org.  The Java SDK can be downloaded from Oracle.

Download An Editor

Groovy can be edited in many free and paid programs.  Some of them are more robust than others and provide things like automatic code completion, color coding, and more advanced features that aren’t likely required at this novice level.  They also increase the complexity for those that are completely new to writing JAVA or Groovy.  If you are interested in this or need a longer term solution, check out these editors.

For those who want to just get started with a simple and supported editor to test some basic code, try the Apache Groovy Console.  The Windows Installer, the documentation, and the SDK are available to download and install.  Once installed, you are ready to go!  If you go to your Start menu in Windows, you will see a folder for the version of Groovy installed.  In that folder click on Start GroovyConsole to open the editor.

Using The Groovy Console

Much of what is done in the Groovy calculations can’t be accessed here.  We don’t have grids, cells, or any of the PBCS  methods that we interact with in a Groovy Calculation.  Groovy can also access the REST API (outside of Groovy Calculations), which opens up the ability to manage PBCS like EPM Automate.  I recently looped through the product catalogue at BestBuy.com and built a hierarchy!  This is a whole other beast, but it is worth mentioning.

Before we jump into testing a script, here are a few things that will be helpful using the Groovy Console.

  • The editor has two panes.  The top pane is where the script is developed and edited.  The bottom pane is where the results of the script are displayed when it is executed.
  • The toolbar has some common functions.  You can open and save your scripts, redo/undo, and execute from icons in this area.

Examples

I find it very helpful as I am learning, to test the logic and the results in this console.  Once validated, it will be moved to the PBCS calculation and used appropriately.  Here are some examples where it might be useful, and hopefully the separation of where to test what is highlighted.

Regex Example

There was a requirement on a form at a recent client where they wanted to accept input.  They used this to setup properties in the HR system.  The HR system could not accept some characters, so the ask was to only allow alphanumeric characters, a space, an underscore, and a dash.  We had to add validation to the run time prompt, as well as when the data was updated in a form.  Not being an expert with regex, I didn’t want to test this in a calculation (update calc, run calc, open job console, expand status, toggle between windows, etc).

So, I opened the Groovy Console and tested there.  The end result is below, but it was much easier to tweak the regex syntax directly in the console, running it, and seeing the result immediately, in one step.  This was easy to see and verify the output was void of any characters that were not allowed.  The length could be compared, pre and post character removal, and was used to stop the save of the data.

String text = "This - text ! has \\ /allot # of % special % characters"
println text
println text.length()
println text.replaceAll("[^a-zA-Z0-9 _-]", "")
println text.replaceAll("[^a-zA-Z0-9 _-]", "").toString().length()
println text.length() == text.replaceAll("[^a-zA-Z0-9 _-]", "").toString().length()

At this point, I proved out the regex functionality.  I can now go back to the Groovy Calculation and use this logic on the variable returned from the PBCS function (whether it be an RTP or a cell value) and remove the invalid characters or test to see if there are any, and act accordingly.  This is what it would look like

String enteredValue = rtps.RTP_NewEmployee.getEnteredValue();
if(enteredValueAdj.length() == enteredValue.length())
{ 
def mbUs = messageBundle(["validation.InvalidChars":"You have entered invalid characters.  Only alphanumeric characters, spaces, dashes, and underscores are accepted."])
def mbl = messageBundleLoader(["en" : mbUs])
throwVetoException(mbl, "validation.InvalidChars", rtps.RTP_NewEmployee)
}
Converting Nested Collections

I was building a Data Map override from a POV, and it wasn’t validating because some of the variables were collections that included a nested collection.  This whole concept was completely new to me, and again, I didn’t want to have to go through 3-5 steps to see if the result was returning a delimited list of members that the Data Map would accept.  Since I had no initial idea how to accomplish this, I searched and found examples that might accomplish what I wanted to achieve.  It took 5 to 10 iterations of examples to get to what I wanted and understand how this worked.  Updating a script in the Groovy Console, running it, and seeing the results in the same window proved much quicker to find a solution.

In the solution below, I created a variable that replicated the variable that PBCS that was returning (a list).  I was able to build out a few lines to eliminate the nested collections and ported this over to my Groovy Calculation.

This proved out that the simple loop below would give me a list I could pass to the Data Map, and was much quicker to solve than trying to do this in PBCS.

def orig_list = [10, 20, [1, 2, [25, 50]], ['Groovy']]
def usable_list = []

orig_list.collectNested([]) { 
 usable_list << it
 } 
println usable_list 
println '"' + usable_list.join('","') + '"'

The result of the executed script created two lines.  At this point, I could use this function in the Groovy Calculation by replacing the orig_list with the object returned from the PBCS function.  I used the usable_list in the Data Map.

[10, 20, 1, 2, 25, 50, Groovy]
“10”,”20″,”1″,”2″,”25″,”50″,”Groovy”

Wrapping Up

These examples are great examples of how we can use a pair of tools to create business logic efficiently.  If you are a seasoned java developer, much of this might seem ridiculous to you and question why one would ever use something outside of PBCS.  I get it.  Now that I know how these two function work, I likely will not use the Groovy Console to write and test this.  But, as I continue to learn more and more, being able to do this in something outside of PBCS has proven invaluable, increased my productivity, and significantly reduced my frustration.

If you are learning, or are an experienced Groovy developer, please share your insights with the community and post a comment!




Adventures In Groovy – Part 11: Accessing Metadata Properties

Introduction

Groovy opens up a lot of things above and beyond performance improvements and improving the user experience.  One example is the possibility to interact with the metadata.  Dimensions and members can be queried for all types of things which can be useful in many situations.  Is the POV at a level 0?  What is the parent of the current POV member?  Does the member exist in another application?  What about pushing data for specific UDAs and dynamically generating the Data Map?  How about dynamically generating the Data Map to ignore dynamic calculated members?  These are just some examples to get you thinking about where this could be useful.

Code Example

This article won’t get into the logic to accomplish the above examples once the property is identified but will explain how to extract properties for its use.  Below is an example of retrieving every property of an account named Regular_Cases.  This iterates through every metadata property and writes it to the log.

// Get the dimension of the member in question
Dimension AccountDim = operation.application.getDimension("Account")
// Get the member 
Member AccountMbr = AccountDim.getMember("Regular_Cases")
// Print the map to the log
println AccountMbr.toMap()
def memberProps = AccountMbr.toMap()
// Print the member name
println AccountMbr.toString()
// Print every property and corresponding property value
for ( e in memberProps ) {
  println "${e.key} = ${e.value}"
}

When this is executed, the following is sent to the log.

println AccountMbr.toMap() produces

{Formula (rGP)=<none>, Plan Type (GP)=true, Solve Order (rGP)=0, Formula (Fin)=<none>, Data Storage (OEP_WFSC)=never share, Time Balance=flow, Formula=<none>, UDA=HSP_NOLINK, Skip Value=none, Variance Reporting=non-expense, Data Storage (GP)=never share, Essbase Name=Regular_Cases, UUID=c842d186-6d83-4b90-8d1e-49474a6a8a1d, Member=Regular_Cases, Data Storage=never share, Data Storage (rFin)=never share, Formula (rFin)=<none>, Aggregation (rWFP)=+, Formula (GP)=<none>, Data Storage (rWFP)=never share, Data Storage (OEP_REP)=never share, Data Storage (rGP)=never share, Data Type=currency, Formula (OEP_WFP)=<none>, Plan Type (rFin)=true, Aggregation (OEP_WFP)=+, Data Storage (OEP_WFP)=never share, Parent=GP_Accts, Two Pass Calculation=false, Aggregation (GP)=+, Plan Type (rGP)=true, Process Management Enabled=true, Plan Type (rWFP)=false, Source Plan Type=GP, Aggregation (OEP_WFSC)=+, Exchange Rate Type=none, Plan Type (Fin)=true, Alias: English=Regular Cases, Plan Type (OEP_WFP)=false, Aggregation (OEP_REP)=+, Solve Order (rWFP)=0, Data Storage (Fin)=never share, Hierarchy Type=dynamic, Allow Upper Level Entity Input=false, Account Type=revenue, Formula (OEP_REP)=<none>, Aggregation (Fin)=+, Aggregation (rGP)=+, Plan Type (OEP_WFSC)=false, Formula (rWFP)=<none>, Formula Description=<none>, Aggregation (rFin)=+, Solve Order (rFin)=0, Formula (OEP_WFSC)=<none>, Solve Order (OEP_REP)=0, Valid For Consolidations=false, Plan Type (OEP_REP)=false}

for ( e in memberProps ) {println “${e.key} = ${e.value}”} produces

Regular_Cases
Formula (rGP) = <none>
Plan Type (GP) = true
Solve Order (rGP) = 0
Formula (Fin) = <none>
Data Storage (OEP_WFSC) = never share
Time Balance = flow
Formula = <none>
UDA = HSP_NOLINK
Skip Value = none
Variance Reporting = non-expense
Data Storage (GP) = never share
Essbase Name = Regular_Cases
UUID = c842d186-6d83-4b90-8d1e-49474a6a8a1d
Member = Regular_Cases
Data Storage = never share
Data Storage (rFin) = never share
Formula (rFin) = <none>
Aggregation (rWFP) = +
Formula (GP) = <none>
Data Storage (rWFP) = never share
Data Storage (OEP_REP) = never share
Data Storage (rGP) = never share
Data Type = currency
Formula (OEP_WFP) = <none>
Plan Type (rFin) = true
Aggregation (OEP_WFP) = +
Data Storage (OEP_WFP) = never share
Parent = GP_Accts
Two Pass Calculation = false
Aggregation (GP) = +
Plan Type (rGP) = true
Process Management Enabled = true
Plan Type (rWFP) = false
Source Plan Type = GP
Aggregation (OEP_WFSC) = +
Exchange Rate Type = none
Plan Type (Fin) = true
Alias: English = Regular Cases
Plan Type (OEP_WFP) = false
Aggregation (OEP_REP) = +
Solve Order (rWFP) = 0
Data Storage (Fin) = never share
Hierarchy Type = dynamic
Allow Upper Level Entity Input = false
Account Type = revenue
Formula (OEP_REP) = <none>
Aggregation (Fin) = +
Aggregation (rGP) = +
Plan Type (OEP_WFSC) = false
Formula (rWFP) = <none>
Formula Description = <none>
Aggregation (rFin) = +
Solve Order (rFin) = 0
Formula (OEP_WFSC) = <none>
Solve Order (OEP_REP) = 0
Valid For Consolidations = false
Plan Type (OEP_REP) = false
Data Storage (GP) = never share

Getting A Specific Property

Typically, there would not be a need to pull every property.  There might be times when having access to these, however, is useful in calculations.  If a currency calculation is being executed, for example, the rate applied is different if the member is a balance sheet account.  Getting one value can be retrieved by building on the above script.

def keyProp = "Account Type"
if(memberProps[keyProp] = "Revenue"
  {do something}
elseif(memberProps[keyProp] = "Balance Sheet"
  {do something}

Wrap Up

This may seem a little worthless at first, but if you think about all the BSO functions (getting UDAs, Account types for VAR functions, and member relation functions) that require this information, mimicking them in Groovy requires access to the metadata properties.  So, don’t underestimate its use for things like variance, currency, and other calculations, that are done outside of Essbase/Planning calculations and member formulas.




Adventures in Groovy – Part 10: Validating Form Data

Introduction

One of the huge frustrations I have with Planning is the fact that you haven’t been able to stop a user from saving data that didn’t validate since Smart View was released and Data Forms could be opened in Excel.  Prior to that, the forms could be customized with JavaScript and the form save could be interrupted and cells flagged.  Well, thanks to Groovy Calculations, it is back, and every bit as flexible.

Example

The following Data Form allows users to add new products and enter financial information.  In this form, 3 rules exist.

  1. The GP Level 2 % has to be between -10% and 30%.
  2. The Regular Cases CANNOT have one month that is more than 50% of the total cases.
  3. If Regular Cases is entered, a corresponding Average Price per Case is required.

When a user fills out the form and violates these rules, the cell background is changed and a tool tip is added.  If violations exists, the form does NOT save any changes to the database.  Before any changes can be committed, all errors have to be corrected.  In this example, all 3 validation rules are violated and noted.  If the user hovers the mouse over the cell with a violation, the tool tip is displayed with the row and column members, and an error message explains to the user what the issue is with the data that is entered.

The Code

The significance of this is huge, but the implementation is rather simple.  It is probably be one of the more basic things created with a Groovy Calculation.  Quite simply, to add a validation error and stop the form from saving, all that has to be done is to add a validation error to the cell.

cell.addValidationError(0xFF0000, “Customer Error Message“,false)

This method accepts 3 parameters.

  1. The first is the color you want the background to change to.  This is the integer value of any color.  Most people are familiar with the RGB code, and this can be retrieved in almost any image editor (even Windows Paint).  There are many free options, like the free option at https://www.shodor.org/ to convert that to a value that can be interpreted in Groovy.
  2. The second parameter is the error message to be displayed in the tool tip.
  3. The third is optional, and defaults to false.  False that it will identify the cell as an error and stop the form from saving.

This will likely becused in a grid iterator, which is how this example was constructed to get the screenshot above.  If the grid iterator object is foreign to  you, read Adventures in Groovy – Part 3: Acting On Edited Cells.  The one function that is void from that article is the crossDimCell method.  This acts like a cross dim (->) in a calculation.  So, it references the POV of the cell and overrides the dimension of the member specified as a parameter.  If multiple differences exist, separate the names with a comma.

def BackErrColor = 16755370 // Light Red
//Loop through the cells on the Regular Cases row
operation.grid.dataCellIterator('Regular_Cases','Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec').each { 
  // Set a variable equal to all 12 months
  def CaseTotal = it.crossDimCell('Jan').data + it.crossDimCell('Feb').data + it.crossDimCell('Mar').data + it.crossDimCell('Apr').data + it.crossDimCell('May').data + it.crossDimCell('Jun').data + it.crossDimCell('Jul').data + it.crossDimCell('Aug').data + it.crossDimCell('Sep').data + it.crossDimCell('Oct').data + it.crossDimCell('Nov').data + it.crossDimCell('Dec').data 
  // Check the cell value to see if it is larger than 50% of the total year
  if(it.data / CaseTotal > 0.5 ) 
    it.addValidationError(BackErrColor, "Cases for a single month can't be more than 50% of the total year cases.", false) 
  // If cases are entered, make sure there is a corresponding price
  if(it.data != 0 && (it.crossDimCell("Avg_Price/Case_Inp").data == 0 || it.crossDimCell("Avg_Price/Case_Inp").data == '#Missing'))
    it.crossDimCell("Avg_Price/Case_Inp").addValidationError(BackErrColor, "A price is required when cases are entered.", false) 
}

// Loop throught the GP input cells and validate the % is in the valid range
operation.grid.dataCellIterator('GP_2_%_Inp','Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec').each { 
  println "$it.MemberNames $it.data"
  if(it.data > 0.3 || it.data < -0.1 ) {
    it.addValidationError(BackErrColor, "GP2 has to be between -10% and 30%.", false) 
 }
}

Form Settings

The one gotcha is that this needs to run BEFORE SAVE.  It makes sense, but I was expecting a similar result as validating a RTP when the Business Rule runs on save, so it took me a good night sleep to recognize that error in judgement.

Why This Is Significant

You may not thing this is a big deal because you can check this in a Business Rule after the data is saved and return an error message requesting the user to change it.  However, the users are as busy, if not more busy, than you are.  There are last minute changes that get slammed in at the end of a forecast or budget cycle.  There is no design doc to go back to and say it is going to take longer and we need a change order.  The CFO won’t accept that as an answer, so things get forgotten or missed.  This example forces valid data (not necessarily accurate) to be entered, and all kinds of things can be checked to make sure human errors don’t turn into huge issues for financial reporting.  Think if you had a product and forgot to put a price.  You could be missing millions, and this type of proactive validation can prevent such things from happening.  Little things like this reduce, or eliminate, fire drills later on in the budget cycle.

Conclusion

There is an infinite number of things that can be accomplished.  Simple things like the above, to extremely complex validation can be added.  Think about ensuring allocated dollars are fully allocated(100%), forcing salaries to be in predefined pay bands for titles, and forcing the results of driver based planning to be within a logical margin.

If you have some examples, please share with the community by posting a comment below.




Adventures in Groovy – Part 9: Ohio Valley OAUG Presentation

I was lucky enough to be invited to talk about the new Groovy Calculation in PBCS and ePBCS at the Ohio Valley OAUG meeting today.  If you have read the Groovy series, you know how strongly I feel about the advancements in Hyperion Planning with the addition of Groovy Calculations.  I want to share the presentation with a wider audience.  This is a functional overview for those who are new to the concepts.  This also introduces readers how to develop their first Groovy Calculation, and provides some examples.






Adventures in Groovy – Part 8: Customizing Data Maps and Smart Pushes

Introduction

Groovy has the ability to execute Data Maps and Smart Pushes.  Data Maps are objects that define the data movement between applications and are accessible globally.  Smart Pushes are Data Maps that are attached to a Data Form and can override the dimensions settings.  They are largely the same thing, but defined separately inside of PBCS.

So, why execute these in Groovy rather than assign to a form and call it a day?

  1. The data push can be customized to push only data that has changed, improving performance.
  2. When a form is saved, all calculations are executed first, and the Smart pushes are executed after all calculations are finished.  If Groovy is used and the data push is done inside a Business Rule, the order of operation can be a
    1. calculation
    2. data push
    3. calculation
    4. data push
    5. etc
  3. Since the Smart Push has a global limit of memory that can be used, and multiple people are running them through form saves, it is critical to make it as small as possible to reduce the probability of hitting that limit and increasing the odds of the Smart Pushes failing.

To date, I see the advantage of running a Smart Push (a Data Map on a form) in that most of the dimensions are already overridden and less overrides are required in the Groovy logic.  There is no difference in performance or the size of what can be pushed between the two when executed from a Groovy calculation.  The advantage of using a generic Data Map is that there is less effort in defining the form level Smart Pushes, and once one Groovy calculation is written to override all dimensions from a Data Map, it can be replicated easily to all required forms.

To understand the memory issues and explanation of how it differs from Data Maps and Smart Pushes, see PBCS Data Map / Smart Push Has Data volume Limits.

Data Map

Executing a Data Map is very simple and can be done in one line of code.

operation.application.getDataMap("Data Map Name").execute(true)

Calling execute() on a DataMap would execute the named Data Map (with no customization) and clearing the target location before pushing data.  Changing the true to false, or removing it, would remove the clear and leave the data as is.

To override the the dimension settings and increase or decrease the scope of what is used, the member names need to be further defined.  Every dimension’s scope can be changed, or just the ones that require a change in scope can be altered.  The following changes the scope for the account, scenario, version, and Entity dimensions.

operation.application.getDataMap("Data Map Name").execute(["Account":"Net Income, Income, Expense", "Scenario":"Budget", "Version":"Working", "Entity":"East Region"], true)

Smart Push

The Smart Push works exactly the same, except the object referenced is a Smart Push, and is obtained through the grid object, not the application.  Again, the likelihood that the Smart Push is further scoped is high, so it is reasonable that the dimensional parameters would be fewer as many of them change based on the POV selected.

operation.grid.getSmartPush("Smart Push Name").execute(["Account":"Min Bonus, Min Salary"])

One additional option is to define a Smart Push from a Data Map in the Groovy script.

operation.application.getDataMap("Data Map Name").createSmartPush().execute(["Account":"Min Bonus, Min Salary"])

Error Trapping

When these are written, it is likely that the Smart Pushes and Data Maps exist.  One extra step to ensure the calculation doesn’t fail is to verify their existence.  For Smart Pushes, verify that it is attached to the form.

//Data Map Example
if(operation.application.hasDataMap("Data Map Name"))
  operation.application.getDataMap("Data Map Name").execute(true)
//Smart Push Example
if(operation.grid.hasSmartPush("Smart Push Name"))
  operation.grid.getSmartPush("Smart Push Name").execute()

Conclusion

Creating variables to use in these functions to change the scope to only the rows and columns that have been edited, the calculation would look like this.  This is where I see the biggest bang for  your buck.  To understand more about using the grid iterator, read Adventures in Groovy Part 3: Acting On Edited Cells.  When a grid has hundreds of rows, only pushing the data that has been edited can make a huge difference in performance.

// Setup the variables to store the list of edited vendors and periods
def lstVendors = []
def lstPeriods = []
// Iterate through only the cells that were changed and create a list of changed vendors and periods
operation.grid.dataCellIterator({DataCell cell -> cell.edited}).each{ 
 lstVendors.add(it.getMemberName("Vendor"))
 lstPeriods.add(it.getMemberName("Period"))
}
// Convert the lists to a comma delimited string with quotes surrounding the members
String strVendors = """\"${lstVendors.unique().join('","')}\""""
String strPeriods = """\"${lstPeriods.unique().join('","')}\""""
// Execute the Smart Push with a change in scope to Vendor and Period
if(operation.grid.hasSmartPush("GP_SmartPush") && lstVendors)
 operation.grid.getSmartPush("GP_SmartPush").execute(["Vendor":strVendors,"Period":strPeriods]

With any luck, you will see significant improvement, unless they change every month of every row!




Adventures in Groovy – Part 7: Validating Run Time Prompts

Introduction

When developing applications in PBCS, there are times when we need to request information from users.  This is especially useful in workforce application, but certainly not limited to them.  With Groovy, we can take validation to the next level.  We can compare inputs to each other, we can limit input, and we can now stop the execution of the calculation until the inputs are validated.  The difference now is that we have the ability to keep the prompts open, and force the user to either enter valid data, or cancel the request.

Validating Hire Date / Termination Date Use Case

Let’s start with a very simple example.  Let us assume we are adding a new employee, and we are asking for a hire and termination date.  A real-life example would require more options, like a name, title, union, and other required inputs.  To simplify this tutorial, we are only entering the hire and termination dates to prove out the validation and show functionality of the Groovy RTP validation logic.

When a user enters a termination date after a hire date and launches the rule, it validates and executes the rule.

When the job is opened in the job console, we see the RTPs and the values entered, and the Job Status is selected, the log shows the values returned to Essbase.


When a user enters a termination date prior to a hire date and launches the rule, it an error is returned and the RTP window doesn’t go away.  At this point, the user has to correct the error, or cancel the execution of the business rule.

In this case, the log shows the business rule failed.

Code

There are multiple objects that are used to accomplish RTP Validation.  The code that processed the above action is the following.

/*RTPS: {RTP_HireDate} {RTP_TermDate}*/
def mbUs = messageBundle(["validation.InvalidDate":"The termination date must be after the hire date."])
def mbl = messageBundleLoader(["en" : mbUs])

// Validate the Rtp values
if(rtps.RTP_HireDate.getEssbaseValue() > rtps.RTP_TermDate.getEssbaseValue())
   throwVetoException(mbl, "validation.InvalidDate", rtps.RTP_HireDate)

// Print the results to the log
println "Hire Date: " + rtps.RTP_HireDate.getEssbaseValue()
println "Term Date: " + rtps.RTP_TermDate.getEssbaseValue()

rtps object

Creating RTPs in Groovy was covered in the previous article.  If you haven’t read that, it would be a good time to take a look, as it explains the basic of this object.  Expanding on the use of the object, we are using some additional methods.  This object has many, including returning the input as boolean, double, strings, dates, members, and smart lists, to name a few.  In this example, we are using getEssbaseValue, which returns the value sent to Essbase and stored.  If there was a need to compare date ranges, we could have used the getDate, and expanded on this with the Groovy date functions to get the days, months, or years between the entered values.  In this simple example, we just want to make sure the term date is greater than the hire date.

messageBundle

The first thing that is requires is to create a messageBundle and messageBundleLoader.  These two objects work together to hold the errors, the error messages, and multiple languages, if required.

The messageBundle is a map that holds all the errors (name and description).  In this example, we only have one error, but more can be added and separated by commas.  The messageBundleLoader holds all the messageBundle objects that represent the different languages.

throwVetoException

When an exception is found, executing this method will initiate an error and cause the RTP validations to fail.  This method requires the messageBundleLoader, and the error to be returned to the user.

Other Use Cases

By now you are probably already thinking of other uses of this.  I can see limiting growth rates, confirming combinations of RTPs (like not allowing salaried people in a union), ensuring that a new employee doesn’t have a hire date prior to the current date, and probably hundreds of other ways to use this.

If you would like to share an idea, please post a comment!

Conclusion

Being able to validate user input prior to executing a calculation and returning the input request to the user is  huge step forward, and just another benefit of Groovy calculations.  We can reduce the number of user errors and improve the user experience.




Adventures in Groovy – Part 6: Converting a POV into a Fix

Introduction

One of the fundamental features of a Groovy calculation is the ability to dynamically grab aspects of a grid, and getting the POV is something that is required to dynamically generate an Essbase calculation.  There are times when the entire POV is required, times when only members from specific dimensions are needed, and situations where only the rows and columns of the edited cells are used to construct effective fix statements.  All 3 of these situations will be discussed.

Use Case (Pulling POV Members)

Many times, the Fix statement of a calculation being built includes all members in a data grid POV.

Code

List<String> povMemberNames = operation.grid.pov*.essbaseMbrName 
String calcScript = """   
Fix("${povMemberNames.join('", "')}")
 [Calculation]
EndFix;"""
Breaking It Down

The operation.grid.pov provides access to all the dimensions in the POV.  From that, dimension names, member names, and other properties that are useful, can be accessed.  When followed by a *, it returns all the dimensions in the POV to a list.  Using the essbaseMbrName instructs the function to return every member in the POV.  The povMemberNames variable stores a list  of all those values.

When building the calcScript variable, ${povMemberNames.join(‘”, “‘)} will return the list, delineated with “,”.  This would return something like membername1″,”membername2″,”membername3.  This obviously is missing surrounding quotes, which is why it is embedded inside the quotes.

Use Case (Pulling Selective Dimension Members From The POV)

Pulling one, or all of the dimension values in a POV, in individual variables for later use, provides the ultimate flexibility.  The following pulls all the members and stores them in a unique variable.  Then, any dimension in the POV can be accessed.  An easy way to accomplish this is to use the find method of a List object to filter the elements.

Code

List<String> povmbrs = operation.grid.pov
String curYear = povmbrs.find {it.dimName =='Years'}.essbaseMbrName
String curCompany = povmbrs.find {it.dimName =='Company'}.essbaseMbrName
Fix((${fixValues(curYear, curCompany))
 [Calculation]
EndFix;"""
Breaking It Down

The first line stores the entire POV collection.  That variable can be accessed and used like any Groovy collection.  By using the find method, items in the collection can be filtered.  povmbrs.find {it.dimName ==’Years’} will return the Years dimension object and one of the properties of that object is the essbaseMbrName.  Using it will return an Essbase friendly member name.

The “fixValues” method converts strings to “Fix friendly” strings that can be used in an Essbase calc script.  Any of the following objects can be inserted into this method.

  • AttributeDimension
  • AttributeMember
  • Dimension
  • Member
  • PeriodDimension
  • RtpValue
  • YearDimension

The result is the dimension member name inside a fix statement.

Rows/Columns

Some of the real efficiencies with Groovy stem from the fact that the edited cells can now be determined and a Fix statement can dynamically be generated to isolate longer running calculation on only the rows and columns that have changed.  In the following example, there are two methods to get the members.

Code

Set<String> periods = []  
Set<String> products = [] 
operation.grid.dataCellIterator({DataCell cell -> cell.edited}, MemberNameType.ESSBASE_NAME).each { DataCell cell ->         
  periods << cell.periodName       
  products << cell.getMemberName("Product")     
  } 

String calcScript = """   
Fix("${periods.join('", "')}", "${products.join('", "')}")
 [Calculation]
EndFix;"""
Breaking It Down

If you are unclear about how to iterate through a grid, read Adventures in Groovy Part 3: Acting On Edited Cells.  Inside the dataCellIterator, the example assigns two list objects with the respective members.  Since this only touches the cells that have been edited, only the periods and products that have been updated will be stored in the respective variables.

You may be wondering if this is truly the most isolated combination of data.  Theoretically, the same product may not have the same edited months.  You could further condense changes in the Fix statement by looping through the rows and creating a fix for every row.  In this example, Period is dense, so doing this would not change the number of blocks.  Depending on the application, taking this extra step might prove more efficient.  Rather than complicate this further, we are assuming this isn’t required.

Every required dimension (Account, Entity, Period, Scenario, Version, and Year) has its own method. cell.periodName returns the Period member of the cell.  All the dimensions have the same naming convention.

For custom dimensions, getMemberName can be used with the required dimension passed as a parameter.  If you want consistency, this method can also retrieve the 6 required dimensions.  cell.getmemberName(“DimensionName“), where DimensionName is an actual dimension, returns the respective member in that dimension.

Just like the previous example, add the variable to the Fix statement with a join and it returns the delimited list of members that have been edited.

  • “${periods.join(‘”, “‘)}” returns the list of Periods that have edited cells, and
  • “${products.join(‘”, “‘)}” returns the rows (or products) that have been edited.

Wrapping Up

One last step that can be added, and should be, is the check to see if any cells have been modified by evaluating the size of one of the lists that is created during the grid iteration.  After the iteration is completed, the following can be added to accomplish this.  If there are no cells edited, the calculation is stopped at the return line and nothing is sent back to Planning/Essbase to execute.

if(products.size() == 0) {
    println("No edited cells found!")
    return
}

Joining the examples above, the fix would look like this.

String calcScript = """  
 Fix("${povMemberNames.join('", "')}", "${periods.join('", "')}", "${products.join('", "')}", ${fixValues(curYear, curCompany))
   [Calculation]
EndFix;"""
return calcScript

Conclusion

If you implement this, you will likely see huge performance improvements.  In the client production applications, I have implemented this in, I see at least a 95% improvement in performance.  This same logic can be used for Data Maps and data movements from ASO to BSO (which we will cover later)

 




Adventures in Groovy – Part 5: Accessing Substitution Variables

Introduction

Accessing Substitution Variables is critical in most calculations, and accessing them in Groovy is a little more complex than it needs to be with not having an API to get them.  Since the SubstitutionVariable is not available, there are a couple ways to get them.  The precursor to this post is three-fold.

  1. Read the Bug Report: Groovy SubstitutionVariable Class Not Functioning  post on Jan 8, 2018 regarding the SubstitutionVariable class availability.
  2. Thanks to Abhi for providing a great alternative.
  3. It may be helpful to read Adventures in Groovy Part 4: Run Time Prompts to understand how to access RTPs in a Groovy calculation.

In my bug report above, I suggested grabbing them via a hidden column or row from a form.  A reader suggested a another way to do this, and I think it is a better way to accomplish it.  Rather than grabbing the substitution variable by adding it to the form and hiding the column/row from the user, Abhi provided a much cleaner approach to working around not having access to the SubstitutionVariable class by using hidden RTPs.

Create Run Time Prompts to Access Substitution Variables

Assume the following 3 variables are required in business rules.  Create a new RTP for each.  The naming convention is irrelevant, but should be considered and be consistent for easy reference in the business rules.  In this read, I have assumed there isn’t an existing RTP with the defaults set to a substitution variable.  Even if there is, it might be beneficial to create ones specifically for this need so future changes don’t impact the values.

Name: subVar_CurMonth
Type: Member
Dimension: Period
Default Value: &v_CurMonth
RTP Text: N/A

Name: subVar_CurYear
Type: Member
Dimension: Period
Default Value: &v_CurYear
RTP Text: N/A

Name: subVar_BudYear
Type: Member
Dimension: Period
Default Value: &v_BudYear
RTP Text: N/A

Business Rule Inclusion

Inside the business rule, the following convention is required to add the variables.

/*RTPS: {subVar_CurMonth subVar_CurYear subVar_BudYear}*/

Set all the RTPs in the Variables tab to set to hidden so the user isn’t prompted for these.  Now, the substitution variables can be referenced.

def varCurMonth = rtps.subVar_CurMonth.toString()
def varCurYear = rtps.subVar_CurYear.toString()
def varBudYear = rtps.subVar_BudYear.toString()

Conclusion

Since these are likely to be used in many rules, it would be beneficial to add these to a script and embed that script into the rules that need to access these.  Any new variable that needs to be included can be added to the script, and all the business rules would then have access to them.  There are a number of ways to do this with Groovy calculations, but the simplest way is to embed it like a non Groovy business rule.  This can be dragged from the left pane, or entered manually.  The syntax is

%Script(name:="script name",application:="application Name",plantype:="plantype name"

If and when Oracle releases the class that provides direct access to sub vars, expect it to be documented here.




Adventures in Groovy – Part 4: Run Time Prompts

Introduction

Groovy provides a very easy way to interact with the user via run time prompts, or RTPs.  These can be linked to dimensions, members, and input.  One of the huge benefits of getting RTPs in Groovy is that the result can be validated, and the calculation can be cancelled if they don’t validate (we will touch on this in a future post).

The Solution

This is one of the easier things to do with a Groovy calculation.  There are two things required.  First, the Groovy calculation must prompt the user to select a value.  This is done by doing the following.

/*RTPS: {RTP_Consolidate_Data}*/

At any point in the script after the above, the value can be used.  If it is going to be used multiple times, it might be easier to set a variable.  Regardless of the approach, the value can be referenced using the rtps object as follows.

 String sRTP
 sRTP = rtps.RTP_Consolidate_Data.toString()

That is all that is required!

Conclusion

Beyond the obvious uses of an RTP, I have started using these for a number of other reasons.

  • On global forms where multiple values may be changed throughout a short period of time and execute long running calculations, like allocations, I have seen benefits of prompting a user with a yes/no smartlist RTP.  If the user has more changes, they may not need to execute the calculation after every save.  This gives them the option.
  • If there is a requirement where some prompts are dependent on other prompts, using RTPs in Groovy gives you the flexibility to validate the combination.  For example, if an employee is set to hourly with a VP title, the prompts can be validated and returned to the user as invalid combinations before the prompts are removed from user view.



Adventures in Groovy – Part 3: Acting On Edited Cells

Introduction

With the introduction of Groovy Calculations this summer, one of the things I use most, especially for applications with data forms that include a large sparse dimension in the rows with suppression on, is the option to loop through cells and identify only the POV on the cells that have changed.  In applications like workforce planning, or product level applications that have hundreds, if not thousands, of possible blocks, isolating only the changed data can have significant impacts on performance.  Normally when a data form is saved, the fix includes all level 0 members of the employee dimension and must run the calculations on all of them, regardless of whether employee changed or not.  Being able to fix on only a row, or the handful that change, give us a significant advantage in user response.

This post will go into how this is executed, and a few use cases to get you thinking about the possibilities.  All of these I have developed and are in a production application.

The Code

Using a grid iterator, with the appropriate parameter, is an extremely easy way to deploy functionality that looks through ONLY the cells that have been changed.

 operation.grid.dataCellIterator({DataCell cell -> cell.edited}).each{
  [actions]
 }

The cell object, and all its parameters, are available inside the loop.  By adding {DataCell cell -> cell.edited}, the loop is isolated to only cells that have changed.  The representative member names, the data value, if it is locked, has attachments, and many other things can be accessed with the example above.

Use Cases

An infinite number of uses are possible, as you are probably already thinking about. If not, the following will probably spark some creativity.

Customizing an Essbase Fix Statement

One of the most significant benefits of this is the ability to be able to dynamically generate a fix statement and filter what is calculated.  Although the calculation on the Essbase side isn’t improved just by using Groovy, the ability to dynamically write the calculation on only what changed is significant, especially when allocations, data pushes, and other longer running processes are required.

Assuming the rows of a data grid include the dimension Product, and Period is in the columns, the following will create variables that will include only the periods and products that have been updated.  These can be used in the string builder that is passed to Essbase.  So, rather than @RELATIVE(“Product”,0) running on all possible products, it can be replaced with “Product 1″,”Product 2”.

The following creates list and string variable for Product and Period.  Every cell that is updated will add the relative product and period to the list variables.  After all the cell values have been read, the two string variables are set to include a unique list of the dimension members, surrounded by quotes, and separated by commas, which are immediately ready to include in the FIX statement.

def lstProducts = []
def lstPeriods = []
operation.grid.dataCellIterator({DataCell cell -> cell.edited}).each{
  lstProducts.add(it.getMemberName("Product"))
  lstPeriods.add(it.getMemberName("Period"))
  }
def strProducts = '"' + lstProducts.unique().join('","') + '"'
def strPeriods = '"' + lstPeriods.unique().join('","') + '"'

The string builder would look something like this.  In the following example, the other variables are pulled from the POV.

def sScenario=povmbrs.find {it.dimName =='Scenario'}.essbaseMbrName
def sCompany=povmbrs.find {it.dimName =='Company'}.essbaseMbrName
def sYear=povmbrs.find {it.dimName =='Year'}.essbaseMbrName

StringBuilder strEssCalc = StringBuilder.newInstance()
strEssCalc <<"""FIX($sScenario,
  $sCompany,
  $sYear,
  $strProducts,
  $strPeriods
)
 Revenue = units * price;
ENDFIX
"""

At this point, the strEssCalc value can be passed to Essbase and executed.  If only 2 products are changed in 1 month, only those 2 cells would be calculated.  If this data form included 12 months and 1,000 products, the calculation would take roughly 1/500th of the time.

Customizing Smart Push

Smart Pushes on forms, depending on the POV, can exceed a threshold of what a user perceives as acceptable performance.  In the 17.11 release, Data Maps and Smart Pushes can now embedded in the Groovy Calculations.  The 2 huge benefits to this are that

  1. the data that is pushed can be filtered to only the data that changes, decreasing the time of the operation, and
  2. the ability control the operation order of when a push runs (for example, calculation, push, calculation, push)

If a data form has a smart push associated to it, it can be accessed and further customized.  If not, data maps can also be accessed, customized, and executed.

One thing I learned from the Oracle development team is that the Smart Pushes have a max memory that can be accessed.  One Smart Push may never hit that limit if it is isolated enough, but we found issues when multiple Smart Pushes were executed a the same time.  We were seeing multiple, and intermediate, failures in the logs.  So, it is even more critical to make these pushes as small as possible to eliminate that issue.

If we reference the example above in the customized fix, we expand on that and apply the same filter to the Smart Push.  The only addition needed is to encapsulate the strProducts variable in quotes.  If nothing is passed, it runs the smart push as it is setup in the form, so operation.grid.getSmartPush(“appname”).execute() would simply execute the same thing as if the Smart Push was set to run on save.

strProducts = """ + strProducts + """
if(operation.grid.hasSmartPush("appname"))
  operation.grid.getSmartPush("appname").execute(["Product":strProducts,"Period":strPeriods])

Validate data

Having the ability to proactively perform data validation is another great addition with Groovy.  Rather than running a calculation, and after the Data Form save returning a message telling the user that they have to change something, or changing it for them, we now can interrupt the data form save and instruct the user to change it before it has any adverse impact on the data.  The following will change the cells that violate the validation to red, add a tooltip, stop the form save, and throw an error message.  Assume we don’t want the user to enter more than 50,000 in salary.  This threshold can point to data in the application, but is hard coded below for simplicity.

operation.grid.dataCellIterator({DataCell cell -> cell.edited}).each{ 
  if(it.data > 50000){
    it.addValidationError(0xFF0000, "Salary is outside of the allowed range.")
  }
}

Conclusion

This is just a taste of what can be done.  As you can see, with the ability to isolate actions on only the dirty cells, we now have opportunities we haven’t had since pre Smart View, and functions are completely new.  The impact on performance is game changing and the ability we now have to interact with a user pre and post save is ground breaking to the possibilities.