Introduction

It is Go Live day, the system goes online, everything seems ok for a while, and then Kerplunk! Thousands of things could have happened and everyone scrambles to figure it out. What went wrong? My first question is, “Was a proper load test completed?” Yes, Load Test.

Typically HP Load Runner is the industrial strength load testing tool used by larger enterprise clients, but often I see things like the team that uses it either does not have the time or resources to get it done for the project, or maybe the project did not have a tool to do any type of load testing; in either case people figure that design and architecture is solid, I mean it is industrial strength software right? So what is there to worry about? Unfortunately, humans created the software and design architecture. So could have this been avoided? Maybe, and maybe not, but if the system was for the most part duplicated in a non-production environment and a well developed load test was used that replicated a near real load seen in production, many of these issues probably would have bubbled up and people would have had time to isolate the issues, figure out a solution, and apply the new changes in production before going live. This is where this article comes in.

So Apache JMeter to the rescue; no more excuses. JMeter is free, well documented, and can do so much more beyond HTTP type of load testing, which I would love to cover and unfortunately I can only focus this article to a basic load test that works with OAM11g. However I hope this article at least gets you started and even excited to learn more about this great tool. In Part 1 I am focusing on the basics of load testing and how to get JMeter to work with OAM11g, but in Part 2 I will add some advanced ideas like generating a more randomized load of User requests and a couple other interesting additions all in providing a more realistic load against OAM11g.

Main Article

About Apache JMeter

Apache JMeter is a free open source desktop application written in pure Java. JMeter is used as a load testing tool to measure performance. In addition to applying a load to a system, JMeter provides Listeners that monitor the load details so that it can even be used to tune a system; e.g. OHS web servers, OAM Servers, LDAP servers, etc. So before you continue download the latest version of Apache JMeter and get it up and running.

Learning by Example

Now there are a lot of blogs and articles out there that show steps to creating a load test plan, but none that I know of showing how to make this work with OAM11g. Even though this article is not specifically about an Oracle product, I think it is well over due to provide an article that at least helps show how to develop a reasonable load test using the free JMeter that will work for OAM11g.

Now to make it even easier for you to learn, I have written this article so that you will use an example I have provided in a zip file, and wrote the article in a format that explains all the important elements of the JMeter plan to make this all work for OAM11g. What I do not do is cover every little detail about all of the elements of the plan, but I do cover most of them. The Apache JMeter site has some great documentation and if you google there are also a lot of great articles that help understand some of the other great features.

The JMeter OAM11g Load Test Example

So let’s get started and download the example OAM11g_LoadTest.zip plan and lets get started. Note that included in the sample JMeter plan is a supporting comma separated file (CSV) file that holds two columns, username and password, more on this later. Once you open up the plan the screen should look like the following graphic.

The Elements of the Plan

As promised this section covers all the important parts of the JMeter Load Test plan I provided. Note that the most critical of the elements to make this work with the OAM11g login is to use the Regular Expression Extractor, more on that later. So without further ado…

The Thread Group

The Thread Group provides a way to configure the number of threads and users that are load tested against your IDM and Application system. You can tweak how fast the total number of users makes requests by changing the value of the Ram-Up Period. As an example if you have 1000 users, then set the Ramp-Up Period to 150 seconds, that starts the initial requests at 6.66 Users per second, 1000 / 150 = 6.667, and after 150 seconds all 1000 users will be opening requests. The Number of Threads are the total number of users the test will run though, but if you set the Loop Count to 2 it would be 2 times the Number of Threads though not all at once hence the term loop. What you configure the Thread Group for is solely based on the type of metrics you are expecting in production.

Common Configuration Elements

The following are what I call common configuration elements. They are located at the top of the load test plan because they provide common settings or functionality used across all the load test elements.

HTTP Cookie Manager

This configuration helps store cookies sent by OAM so that when JMeter gets the necessary OAMAuthnCookie or ObSSOCookie depending on whether it is a 11g or 10g Webgate respectively, all the necessary cookies will be stored in JMeter so that SSO can be accomplished. Note in the plan provided the option Clear cookies each iteration has been checked so that each new request simulates a User closing their browser thereby requiring to get new cookies on a subsequent new thread.

Common Configuration Elements

HTTP Cookie Manager

HTTP Cache Manager

This configuration will cache resource elements like URLs in JMeter’s memory so that it is not always reaching out for the same URL over and over. The purpose of this is to simulate how many browsers improve performance by caching requests. Note I have also checked Clear cache each iteration so that cache is not reused for each new thread request.

User Defined Variables – Host Variables

This configuration is used to simplify making hostname and port changes across all the HTTP Request samplers that use a host and port. Obviously in my plan I provided your hostname or port will not be the same, or maybe your hostname changes in your test environment. So instead of going through all the HTTP Request elements throughout the plan and updating the Server Name or IP value individually one by one, you can easily make the change in one place. The reason this works is because for all the HTTP Request elements those values are populated with a dynamic variable like ${app_host}, then in the Host Variables table the variable Name “app_host” gets replaced by the actual Value you included in the configuration element. This is a real time saver.

CSV Data Set Config – Login User Data

This configuration provides a way to include an external Filename <some filename>; in the example I include the Users.csv. The User.csv file I provided has two columns, username and password. Alternatively if all the passwords in the OAM Identity Store LDAP server are the same, you could instead add the password as a variable in the Host Variables User Defined Variables element and give the variable Name say “password”, and then enter in the Value field the actual password. The second password column in the User.csv file can now be removed. There are other options too, so feel free to experiment and learn.

The Simple Controller

Within the Logical Controller in JMeter is an option called the Simple Controller, which provides a way to organize all types of elements like HTTP Requests samples. The Simple Controller really does nothing more than provide for a way to logically group elements of the load test plan. For example I used three Simple Controllers and named them Login, AuthZ, and Logout to represent the three functional actions of my Load Test plan. As you build the complexity of the plan you will appreciate breaking the functionality of the test plan into logical groups using this technique. Feel free to expand on the idea and add more Simple Controllers if you feel it is necessary. The following gives more details on each group I created to help better understand what I am trying to accomplish.

Login Controller

The Login Simple Controller is meant for all actions that have to do with a User logging into OAM. The following are explanations of the elements.

HTTP Request – Portal Request

Modify the Path parameter URI to point to a protected HTTP resource. The protected resource should redirect JMeter to the initial login page.

Regular Expression Extractor – Extract request_id (Critical for OAM11g)

This comes from one of the Pre-Processors elements. The Regular Expression Extractor is used to extract the OAM 11g querystring parameter called request_id; this was not required in OAM10g so if you happen to be load testing OAM10g you can disable this element. The element basically looks for the existence of the request_id querystring parameter and then extracts the value, which happens to be different each time the login page is requested, and then inserts it into the login post. Without posting the request_id the OAM login would fail. I also want to bring to your attention that for some reason if you use the out-of-the-box OAM11g login page, the request_id querystring seems to not be present in the HTTP trace. The only way I was able to get this test plan to login properly using JMeter was to create an external form login. If you are not sure how to do this, Brian Eidelman wrote a great blog on this very topic, “External Custom Login Forms with Oracle Access Manager 11g”.

HTTP Request – Login Submit

An important configuration for this element is the path, the value needs to be /oam/server/auth_cred_submit so that JMeter will submit all the credential parameters to the OAM11g login just like a person submitting their credentials in a login form. Note that in the Parameters table all the additional elements besides username and password are included; e.g. request_id, displayLangSelection, and Languages. Please note that the request_id is a dynamic variable ${request_id}, because the actual value that gets populated during the test comes from the Extract request_id element that was mentioned earlier.

Authorization Controller

This is another Simple Controller and contains all the HTTP authorization requests that happen after login. After the Login actions happen from the Login Simple Controller, JMeter has the proper OAM SSO cookies that are temporarily stored in the HTTP Cookie Manager. So in this logical Simple Controller, I make all my URL requests simulating authorizations. In the example I provided there are only three sample HTTP Requests and each Path URI is simply “/”, so please modify each HTTP Request per your load test setup and in fact do not be shy, add more and even elaborate on other options.

HTTP Request – Portal Home
HTTP Request – My Account
HTTP Request – Info Page

Logout Controller

This simple controller contain all the actions required to logout and also verify the logout was successful. Since a typical User will complete a logout, it is important to include this logical controller as part of the load test life cycle.

HTTP Request – Logout Request

Note that the Path parameter needs to point minimally to the /oam/server/logout URI, but in my example I have used the URI value /oam/server/logout?end_url=/logout.html. The end_url is a special parameter that tells OAM to return the User to a custom landing page after the logout. In my setup I have a custom logout.html page on the same host as the custom login.html page. For your setup feel free to remove this extra parameter or modify it as necessary per your setup. It is important to check your OAM11g setup to know how it has been configured to properly logout of the application from which you can then modify the JMeter plan as needed.

HTTP Request – Logout Landing

This element just requests the final logout page. Note that underneath this element is a Response Assertion, which determines if the logout was successful, more on that later.

Response Assertion – Validate Logout

This element is configured for the Main Sample Only, Document (text), and Contains, where the pattern match is “You are now logged out” based on what is displayed in my logout.html page. However, in your case you may want to look for something else that is seen in the final logout page. Basically what the Response Assertion does is it can look at content seen in a page and if valid it will be successful, if not it will show up as red in the View Results Tree and provide a negative response. Take a look at the options in the Response Assertion; it could be text in the HTML page, a URL, or other. Please modify the Response Assertion element per your environment logout page action so that it can determine if the logout was successful. If you want, you could also disable this element.

Listeners

Listeners are elements that provide a way to monitor events during a load test. In the load test example I provided I just used two of the following, but feel free to experiment with other Listeners to learn more.

View Results Tree

This element is very useful because it records the requests and responses from each Sampler event during a load test and displays exactly what the client would see in an Internet browser.

Aggregate Report

This element records a nice comprehensive table of responses like errors, throughput, median, etc. seen in each thread request. As you review the report you can look at each column of each row to help identify bottlenecks or successes. There is even an option to save the results to a file so that iterative reports can be compared to determine positive and negative improvements.

Test the Load Script

Now that you have tweaked the example JMeter load test plan per your environment it is time to test.

Before running the test I want to point out some words of caution. JMeter will consume as much memory as it needs without concern of how much memory in the computer it is running on. Therefore, if you enter a high value in the Number of Threads, your desktop can potentially come to a grinding halt or worse freeze. If you want to send a high volume of requests to the system in the case you need to simulate thousands of requests, it is a good idea run several JMeter instances on more than one desktop and simultaneously run them all at the same time. Since JMeter is written in pure Java, you can run it on any platform that supports the Java JRE 1.6x or higher.

To start it is a good idea to run a light load from a single JMeter instance to just test the plan. Before kicking off the load test, select the View Results Tree and then to start the plan click the green right arrow from the top menu to start the load test; see the illustration below. You should now start to see requests populate the results tree. Once the test is completed review the results in the Aggregate Report. If you want to run the test again use the double broom button to clear everything.

As a refresher, I had mentioned early on that in the Thread Group element you can change various parameters to send all types of loads. Unfortunately give you any general load amounts or specific settings, but just so I don’t let you down and hopefully make this article as complete as I can make it, I am including the next section that touches a bit some general guidelines.

Beyond Apache JMeter

I saved the boring stuff for last; you are welcome. I wanted to take a step back now and talk a little bit about load testing tips in general. The following are things I am aware of based on years of experience, but I would never say these are the golden rules to go by though it is at least I feel are some good guidelines.

1. Use Realistic Metrics

Load testing has been around for years, but all good tests are based on accurate estimates based on real traffic. Do not guess your metrics to load test with, that is simply a bad idea and where people usually get into trouble. Real load pattern metrics that come from production will be made up of average page views, session duration, page size, average and peak loads, and basically data to help simulate what an average user would do logging in and completing some actions. The metrics should also determine how many concurrent logins are seen on average during peak periods. In fact average loads in general are good to know. One reason is that average loads are important during longevity load tests because often a system may hold up gracefully during a peak load, but act differently over longer periods of time even at average loads. Also be realistic, for example if there are peak loads of say 5000 unique concurrent logins an hour, it is a bad idea to just load test with 100 unique concurrent logins per hour even if you push the requests at a higher rate because the software caching can skew the expected results and when you go to production the difference between 100 concurrent logins per minute versus 5000 concurrent logins per hour can make a difference. The result is you may end up being unprepared if the Go Live topples over because of unrealistic load tests.

2. Stress Test all Actions; Login, Authorization, and Logout

Good load testing should simulate the full life cycle of a User using the system. For example login, authorization actions, and logout patterns. I have seen to many times where authentication patterns are only stress tested, and believe that is good enough to simulate the real production system based on those tests. Based from my experience, as a general rule authorization requests account for about 75% percent of the load. In fact I have seen clients that have discovered authorization requests can go even higher at more than 80% percent of the load and that load can even be more compounded by things like Policy Responses in the OAM Policies that send lots of header, cookie, etc. All that additional data adds to the load of the system. Therefore load tests should be made as close to real life as possible by iterating all the basic actions of a life-cycle, logging in, accessing pages, entering data if possible, and logging out. The closer to realistic load patterns that can be designed into the load test, the greater chances issues will bubble up in the non-production system so that you have a chance to correct them before going live.

3. Monitor All Components

Load testing is just a part of the testing phase so don’t think that just sending a big load against your system and it hums right along with flying colors it is all done. Monitoring all components is also critical. There are always cases where certain tiers may not be scaled enough or little problems bubble up and the only way to determine that is to monitor as much as possible. If problems arise, the solution could be as simple as tuning, and in some cases scaling up or even out. In a worse case scenario maybe the design needs to be reworked. The good news is that proper load testing identifies all these things before you go live.

4. Document the Results

This may seem obvious, but documentation seems to be one of those things no one wants to do. It is a good idea to create some type of document with tables to record necessary results in a historical manner. This provides a way to present of all the negative and positive results to determine progress and if the system is ready for prime time. The document should also have ways to record potential issues, bugs, etc. so that those things can be addressed and the load tests can be run again once the defects are corrected. Once all the issues are worked out you can at least show that due diligence has been done before going live. As a benefit if issues do bubble up in production, you have a tool that can help potentially reproduce the problems in a non-production environment so that corrections can be tested versus trying to do it in a live system.

5. How Long will Load Testing Take?

Typically a Project Manager will want to know how much time is needed for the load testing line item. Great question, but this is a tricky question because the exercise of setting up the load test and executing it can mostly be estimated. However when problems bubble up during a load test it is much more difficult to determine in the future what problems will arise and even more challenging how long it will take to correct for the defect especially if it turns out to be a bug. If possible, add enough buffer time to provide some comfort to account for future issues. Good luck with that. If you can have some buffer time, and you are lucky enough to gain some of that time back because things go smoother than expected, you will look like a Rock Star. It is better over estimate and over deliver than to over commit and under deliver. Easier said than done.

Summary

In summary JMeter is a really great load testing tool for OAM11g. It can not only provide a way to simulate a realistic load test, but also a way to stress the system while making tweaks to tuning parameters to determine positive and negative results. JMeter can also be used for other things beyond OAM like the LDAP, Web Service Calls, JDBC, JMS, etc. Most importantly allow enough time to account for issues that bubble up during the testing because there is no way to see into the future. I hope the example plan provides a great start and you go beyond what I have provided. Happy testing!