{"id":2531,"date":"2019-12-06T17:30:50","date_gmt":"2019-12-06T12:00:50","guid":{"rendered":"http:\/\/blog.tenthplanet.in\/?p=2531"},"modified":"2026-03-03T10:13:07","modified_gmt":"2026-03-03T10:13:07","slug":"auditing-pentaho-setup-using-pdi","status":"publish","type":"post","link":"https:\/\/tenthplanet.in\/blogs\/auditing-pentaho-setup-using-pdi\/","title":{"rendered":"Auditing Pentaho+ Setup using PDI"},"content":{"rendered":"<div class=\"wpb-content-wrapper\"><p>[vc_row kd_background_image_position=&#8221;vc_row-bg-position-top&#8221;][vc_column][vc_column_text]<\/p>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>Problem statement:<\/b><\/span><\/h2>\n<p class=\"western\"><span style=\"font-size: large\"> One cannot simply upgrade your Pentaho+ to a higher version just like that. It takes detailed effort on each phase of the upgrade. Any simple mistake can cause enormous\u00a0losses for your company. Even though we know the general steps to be followed during upgrade we might miss a few key points. In effect, we cannot downgrade your version and retrieve\u00a0your data\/config which you have lost due to small negligence.<\/span><\/p>\n<p class=\"western\"><span style=\"font-size: large\"> Apart from that we also need to do the auditing before the Pentaho+ upgrade. This can be a tiresome process for people who are not aware of the architecture. How can we identify those criteria? How can you pass in your auditing of the Pentaho+ setup? Is it easy for a beginner to do this? You can find a solution to all these queries here. <\/span><\/p>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>Solution :<\/b><\/span><\/h2>\n<p class=\"western\"><span style=\"font-size: large\">With regard to the above problem, we came up with \u201cChimney\u201d(Pentaho+ Audit Tool) &#8211; a detailed report on what changes to be done during the upgrade with efforts and suggestions. This helps you to calculate your possibility for a smooth version upgrade. Gives you an idea on your What? Why? How? Questions.<\/span><\/p>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>What Chimney Does?<\/b><\/span><\/h2>\n<p class=\"western\"><span style=\"font-size: large\">A dynamic report will be generated based on the customer with the following category taken into consideration.<\/span><\/p>\n<ul>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">Infrastructure<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">Authentication <\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">Pentaho+ Repository and Security<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">Pentaho+ Data Integration (PDI)<\/span><\/p>\n<\/li>\n<\/ul>\n<p class=\"western\"><span style=\"font-size: large\">You can expect the answers to the following questions on Chimney.<\/span><\/p>\n<ul>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">How much effort needed to do this change?<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">What are the skills required to proceed?<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">What will happen if do not take this into consideration?<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">What challenges will I face when I proceed with this idea?<\/span><\/p>\n<\/li>\n<\/ul>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>How Chimney Does?<\/b><\/span><\/h2>\n<p class=\"western\"><span style=\"font-size: large\">Now let&#8217;s dive into some technical stuff on how we get those details in the form of a report. Do not worry as we will cover only a few topics here. For more details please visit our website.<\/span><\/p>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>1: Depreciated Steps<\/b><\/span><\/h2>\n<p class=\"western\"><span style=\"font-size: large\"> What do you mean by the term Depreciated? &#8211; In Pentaho+ upon every upgrade, they will be few enhancements in the steps as well as removing them to replace it with an alternate step. These steps are called as Depreciated Steps. Now your mind would raise a question as \u2018 How to find them? \u2018<\/span><\/p>\n<p class=\"western\"><span style=\"font-size: large\">We have the list of steps you need to zoom in while you upgrade to the latest\u00a0version.<\/span><\/p>\n<table style=\"height: 293px\" border=\"0\" width=\"759\" cellspacing=\"0\">\n<colgroup span=\"3\" width=\"197\"><\/colgroup>\n<tbody>\n<tr>\n<td colspan=\"3\" align=\"center\" valign=\"middle\" height=\"17\"><b>Depreciated\/Changed steps<\/b><\/td>\n<\/tr>\n<tr>\n<td align=\"left\" valign=\"bottom\" height=\"17\">Avro\u00a0input<\/td>\n<td align=\"left\" valign=\"bottom\">SAP input<\/td>\n<td align=\"left\" valign=\"bottom\">XML Input<\/td>\n<\/tr>\n<tr>\n<td align=\"left\" valign=\"bottom\" height=\"17\">Text file input<\/td>\n<td align=\"left\" valign=\"bottom\">OpenERP object delete<\/td>\n<td align=\"left\" valign=\"bottom\">Example\u00a0step<\/td>\n<\/tr>\n<tr>\n<td align=\"left\" valign=\"bottom\" height=\"17\">Greenplum Bulk Loader<\/td>\n<td align=\"left\" valign=\"bottom\">OpenERP object input<\/td>\n<td align=\"left\" valign=\"bottom\">Text file output<\/td>\n<\/tr>\n<tr>\n<td align=\"left\" valign=\"bottom\" height=\"17\">Aggregate Rows\u00a0Step<\/td>\n<td align=\"left\" valign=\"bottom\">OpenERP object output<\/td>\n<td align=\"left\">IBM Websphere MQ Producer<\/td>\n<\/tr>\n<tr>\n<td align=\"left\" valign=\"bottom\" height=\"17\">Get Previous Row Fields<\/td>\n<td align=\"left\" valign=\"bottom\">Palo cell input<\/td>\n<td align=\"left\" valign=\"bottom\">JMS Consumer<\/td>\n<\/tr>\n<tr>\n<td align=\"left\" valign=\"bottom\" height=\"17\">Google Analytics Input<\/td>\n<td align=\"left\" valign=\"bottom\">Palo cell output<\/td>\n<td align=\"left\" valign=\"bottom\">JMS producer<\/td>\n<\/tr>\n<tr>\n<td align=\"left\" valign=\"bottom\" height=\"17\">LucidDB Bulk Loader<\/td>\n<td align=\"left\" valign=\"bottom\">Palo dim input<\/td>\n<td align=\"left\">IBM Websphere MQ Consumer<\/td>\n<\/tr>\n<tr>\n<td align=\"left\" valign=\"bottom\" height=\"17\">Streaming XML Input<\/td>\n<td align=\"left\" valign=\"bottom\">Palo dim output<\/td>\n<td align=\"left\" valign=\"bottom\">Script<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p class=\"western\"><span style=\"font-size: large\">Compare them against the list of steps present in the transformation file (.ktr) we can get the depreciated steps available if any.<\/span><\/p>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>Solution :<\/b><\/span><\/h2>\n<p class=\"western\"><span style=\"font-size: large\"> Below KTR helps you to find the Depreciated step if you have used any. You can provide the directory path in the XML input step which helps you to read all the KTR files and list the name of KTR and step which needs to be looked into.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2573\" src=\"https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/2019\/12\/img_1-300x42.png\" alt=\"\" width=\"779\" height=\"109\" srcset=\"https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/img_1-300x42.png 300w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/img_1-768x107.png 768w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/img_1.png 926w\" sizes=\"auto, (max-width: 779px) 100vw, 779px\" \/><\/p>\n<p class=\"western\">This is the base transformation for all the auditing we do while upgrading Pentaho+ with respect to the PDI section.<\/p>\n<p><b style=\"font-size: large\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2539\" src=\"https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/2019\/12\/challenges-300x101.png\" alt=\"\" width=\"724\" height=\"244\" srcset=\"https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/challenges-300x101.png 300w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/challenges-768x258.png 768w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/challenges.png 860w\" sizes=\"auto, (max-width: 724px) 100vw, 724px\" \/><\/b><\/p>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>Challenges :<\/b><\/span><\/h2>\n<ul>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">We cannot use the alternative step every time as the logic differs. <\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">The old configuration details cannot be completely used in the new one.<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">We might need to change the flow of the transformation in some cases like MQ producer and MQ Consumer.<\/span><\/p>\n<\/li>\n<\/ul>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>2: Metadata Injection Support<\/b><\/span><\/h2>\n<p class=\"western\"><span style=\"font-size: large\"> One of the key features of Pentaho+ is the Metadata Injector step. This was initially supported by only a few steps in the transformation. Due to this many difficulties were faced at the time of designing a dynamic architecture. Later on, Pentaho+ introduced this facility for many steps upon every upgrade.<\/span><\/p>\n<p class=\"western\"><span style=\"font-size: large\">But our concern lies in what are those steps? How can we use that to make our existing solution\/logic dynamic?<\/span><\/p>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>Solution :<\/b><\/span><\/h2>\n<p class=\"western\"><span style=\"font-size: large\"> For all those above questions we have the solution in the previous KTR where we can filter out the steps we needed. In addition, it helps you to find the step which does support Metadata injection before but the feature added in the latest version.<\/span><\/p>\n<p class=\"western\"><span style=\"font-size: large\"> For example, the below step \u2018String Cut\u2019 do not support Metadata Injection in version 7.1 but it does in the latest version 8.3. You can use this feature to dynamically cut strings with different logic under different scenarios.<\/span><\/p>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>Challenges :<\/b><\/span><\/h2>\n<ul>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">Even after using Metadata Injection we can see some performance issue.<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">Not all the Metadata injection can be used for making your KTR dynamic.<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">There are few steps where Metadata injection is allowed but not for all fields in it.<\/span><\/p>\n<\/li>\n<\/ul>\n<h2 class=\"western\"><span style=\"font-size: large\"><b>3: Authentication Provider<\/b><\/span><\/h2>\n<p class=\"western\" align=\"left\"><span style=\"font-size: large\"> In this final\u00a0part of the blog, we will see the role of an Authentication provider in Pentaho+ Upgrade. Every time we upgrade we will have the below default configuration.<\/span><\/p>\n<ul>\n<li>\n<p class=\"western\" align=\"left\"><span style=\"font-size: large\">Security.properties \u2013 Default security Provider \u2013 Jackrabbit<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\"><span style=\"font-size: large\">ApplicationContext-spring-security.xml \u2013 Authentication provider &#8211; <\/span><span style=\"color: #000000\"><span style=\"font-family: Liberation Serif, serif\"><span style=\"font-size: large\">\u201c<\/span><\/span><\/span><span style=\"color: #000000\"><span style=\"font-family: Liberation Serif, serif\"><span style=\"font-size: large\">anonymousAuthenticationProvider\u201d (or) \u201cdaoAuthenticationProvider\u201d<\/span><\/span><\/span><\/p>\n<\/li>\n<\/ul>\n<p class=\"western\"><span style=\"color: #000000\"><span style=\"font-family: Liberation Serif, serif\"><span style=\"font-size: large\">This is the least level of security that Pentaho+ provides but most of the use-case does not handle the same. We do customization to it and get enhanced authentication.<\/span><\/span><\/span><\/p>\n<p class=\"western\"><span style=\"color: #000000;font-family: Liberation Serif, serif;font-size: large\">We need to identify those customizations and suggest the changes as well as the impacts on the existing one.<\/span><\/p>\n<h2 class=\"western\"><span style=\"color: #000000\"><span style=\"font-family: Liberation Serif, serif\"><span style=\"font-size: large\"><b>Solution :<\/b><\/span><\/span><\/span><\/h2>\n<p class=\"western\" align=\"left\"><span style=\"color: #000000;font-size: large\">The below KTR helps you to find the existing configuration in your system and compare\u00a0against the default one and produce the following result.<\/span><\/p>\n<ul>\n<li>\n<p class=\"western\" align=\"left\"><span style=\"color: #000000\"><span style=\"font-size: large\">Impacts \u2013 (Example &#8211; Less privacy on the repository as it is folder\/file-based)<\/span><\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\" align=\"left\"><span style=\"color: #000000;font-size: large\">Suggestions \u2013 (Example \u2013 Use Multi Authentication for enhanced security)<\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\" align=\"left\"><span style=\"color: #000000\"><span style=\"font-size: large\">Efforts &#8211; ( Example \u2013 Total time required for the upgrade is 3-5 weeks)<\/span><\/span><\/p>\n<\/li>\n<li>\n<p class=\"western\" align=\"left\"><span style=\"color: #000000\"><span style=\"font-size: large\">Benefits \u2013 (Example &#8211; Isolated user\/role management as per multi-authentication provider)<\/span><\/span><b style=\"font-size: large\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2540\" src=\"https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/2019\/12\/authproviderpng-300x176.png\" alt=\"\" width=\"769\" height=\"451\" srcset=\"https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/authproviderpng-300x176.png 300w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/authproviderpng-1024x600.png 1024w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/authproviderpng-768x450.png 768w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/authproviderpng.png 1175w\" sizes=\"auto, (max-width: 769px) 100vw, 769px\" \/><\/b><\/p>\n<\/li>\n<\/ul>\n<h2 class=\"western\" align=\"left\"><span style=\"font-size: large\"><b>SAMPLE REPORT : <\/b><\/span><\/h2>\n<p align=\"left\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-2542 alignright\" src=\"https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/2019\/12\/sample-report-171x300.jpg\" alt=\"\" width=\"471\" height=\"826\" srcset=\"https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/sample-report-171x300.jpg 171w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/sample-report-768x1345.jpg 768w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/sample-report-877x1536.jpg 877w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/sample-report-1169x2048.jpg 1169w, https:\/\/tenthplanet.in\/blogs\/wp-content\/uploads\/sites\/21\/2019\/12\/sample-report-scaled.jpg 1462w\" sizes=\"auto, (max-width: 471px) 100vw, 471px\" \/><\/p>\n<p align=\"left\"><a href=\"https:\/\/tenthplanet.in\/pentaho\/pentaho-upgrade\/\" target=\"_blank\" rel=\"noopener\">For more details, do visit our website&#8230;&#8230;&#8230;&#8230;&#8230;.<\/a><\/p>\n<p>[\/vc_column_text][\/vc_column][\/vc_row]<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>One cannot simply upgrade your Pentaho to higher version just like that. It takes detailed effort on each phase of upgrade<\/p>\n","protected":false},"author":23,"featured_media":2702,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[424],"tags":[569,570,571,503,501],"class_list":["post-2531","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-pentaho","tag-auditing","tag-data-platform","tag-pentaho-audit","tag-pentaho-data-integration","tag-pentaho-upgrade"],"acf":[],"_links":{"self":[{"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/posts\/2531","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/users\/23"}],"replies":[{"embeddable":true,"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/comments?post=2531"}],"version-history":[{"count":0,"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/posts\/2531\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/media\/2702"}],"wp:attachment":[{"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/media?parent=2531"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/categories?post=2531"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tenthplanet.in\/blogs\/wp-json\/wp\/v2\/tags?post=2531"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}