.png)
Heuristic Evaluation
IBM Cloud Pak: Business Automation
IBM Cloud Pak® for Business Automation is a set of integrated market-leading software designed to help you solve your toughest operational challenges. This software consist of actionable AI-generated recommendations, built-in analytics to measure impact, and business-friendly low-code tooling.
Background
At the request of Senior Vice President of IBM Cloud and Data Platform, this heuristic evaluation aimed to identify barriers and implement solutions to enhance the trial user experience for IBM's Cloud Paks across all stages of the user journey: discovery, learning, registration, onboarding, initial use, and purchase. Our specific focus was on the IBM Cloud Pak for Business Automation page, where we were tasked with mapping the current user journey for the discover, learn, try, and demo stages.
Stakeholders
The automation, marketing, analytics, SaaS, design, and research teams collaborated to establish research objectives, and I took full responsibility for leading the research study.
Objectives
Research Objectives
1. Evaluate the Discovery Process
Assess how users initially discover the Cloud Pak for Business Automation marketing page within the context of a predefined user scenario.
2. Understand Information Navigation
Investigate how users navigate the marketing page to find additional information about IBM's Cloud Pak for Business Automation.
3. Assess Product Interest and Engagement
Determine if users express a desire to test run the product and explore how they discover and engage with the trial experience.
4. Map User Journey Flow
Analyze the user journey through the "Discover, Learn, Try, and Demo" stages, identifying any barriers or pain points in each phase.
Business Objectives
1. Improve User Discovery and Engagement
Enhance the marketing page to facilitate easier discovery and navigation, leading to higher user engagement with IBM’s Cloud Pak for Business Automation.
2. Optimize Information Delivery
Ensure that critical information about the Cloud Pak for Business Automation is easily accessible and understandable, thereby improving user comprehension and interest.
3. Increase Trial Participation
Streamline the process for users to discover and engage with the SaaS trial experience, aiming to increase the number of users who test run the product.
4. Refine User Experience Across Journey Stages
Identify and implement improvements across the "Discover, Learn, Try, and Demo" stages to provide a seamless and intuitive user journey, ultimately driving higher conversion rates and customer satisfaction.
Research Methodology
Method: Heuristic Evaluation + User Testing
Participants: 12 users
User Roles: Customer Operations Manager, Senior Manager, Marketing Operations Manager, Business Analyst, and so on.
Principle: Jakob Nielsen's 10 general principles
User Scenario:
"Imagine you want to find a business automation platform to evaluate which solutions address your organization's workflow orchestration, decision managing, and task automation so that you can create and manage AI-directed workflows to automate business processes, use AI to gain understanding and better governance over your enterprise content, and so on."
Severity Scale
Utilized to assess the severity of usability issues throughout the user journey
Key Findings
Users needed constant guidance when accessing the trial though walk me tour was provided.
Upon logging in, users are greeted with a walk-through tour. However, without clear instructions, they may still feel uncertain about the next steps to take. Majority of users expressed dissatisfaction with the walk-through tour, stating that they were expecting a more interactive and engaging trial experience.
-
"I want a simplified version of the whole experience. I want a step-by-step guide to walk me through creating an automated workflow. I need a simple workflow teaching me how to create and a feel good perspective from the tutorial after creating your first automated workflow. That would encourage people to see how simple it is to use" - Business Analyst
Recommendation: When arriving onto the demo trial, users should be given a thorough step-by-step tour guide to walk them through.
Constant navigation issues occurred through the trial experience as multiple links brought users out of context.
Confusing Demo Walk-through: Users find the demo walk-through confusing due to its complicated layout, which seems to present three separate tasks. It then directs them to a complex slide presentation. This layout leads users to mistakenly believe each box represents a distinct demo, causing them to complete only one task, such as "Launch the Demo Dashboard." Only one user followed each slide thoroughly.
"Getting Started" Repeats Walk-through Tour: Clicking on "Getting Started" leads users to the same walk-through tour that appears when they sign in, further adding to their confusion.
Due to excessive content, users misunderstood the concept of "Paks" and were unclear about what is included in the IBM Cloud Pak for Business Automation.
The marketing page contains excessive text, prompting users to navigate through various links to understand the product. At the bottom of the page, there is a list of different products that may or may not be part of the Business Automation platform. Consequently, users may become confused and not fully grasp what is included with the platform. This highlights the need for a clearer and more effective presentation of the bundled products.
The signup process of the Trial Signup had concerning questions, related to issues with filling out and signing up for an account.
Users struggle to understand the specific services, integrations, and apps included with the platform purchase because the marketing page does not clearly outline the products that are part of the Business Automation bundle. This leaves users unsure of what is included and what software they will have access to.
-
"Is it a package of a product or one product? Do you have to buy a package to get the automation or is it just one product?" - Marketing Operations Manage
Unclear pricing structure: There’s no set number that shows the cost of the platform. Though “Pricing” can be seen at the top as a tab, it doesn’t clarify users with the pricing structure.
The marketing pages contain an overwhelming amount of text, making it difficult for users to comprehend the information.
Additionally, users were frustrated with the excessive scrolling required to navigate the content. While the use cases were appreciated, users found the static images unengaging and requested more interactive content, such as videos or GIFs, to better showcase the product.
-
"Page feels very static. I'm more likely to return to sites if they are engaging. Just having text and link just feels not very interactive. I'm very interested in this service but its not very engaging overall. The service is engaging and that's what going to keep me coming back. But the content is super not engaging." - Business Analyst
Recommendations
Enhanced trial experience
To enhance the trial user experience, consider offering a more comprehensive walk-through that consolidates the step-by-step processes of each use case into a single document.
Additionally, we suggest adding a checklist feature to help users track their progress as they complete each task.
Furthermore, incorporating a chat box or other forms of on-demand assistance can provide guidance and support when users encounter difficulties at any stage of their experience.
Visual Clarity
To make it clearer for users what comes with the Business Automation platform, visually show a side-by-side comparison of the different products and services included.
Additionally, create a pricing table to display various options and premiums, making it easier for users to compare this product to competitors and determine the best fit for their company.
Enhanced Trial Signup
To improve the trial signup page, add images and reduce the amount of text to make it look more appealing and modern.
Additionally, provide immediate access to the trial rather than making users wait for 48 hours.
Visual Engagement
To enhance the user experience, transform excessive text content into visual elements such as images or GIFs to increase user engagement.
Remove any unnecessary text and focus on providing clear and concise information about the product.
System Usability Score - Rating
After calculations, users rated the trial experience with a SUS Score of 46.5.
Adjective Rating: Awful
Grade: F
Next Steps and What We've Learned: After the study, we created journey maps and personas for individuals navigating the Cloud Pak for Business Automation steps. This highlighted significant flaws in the overall experience, prompting the team to develop roadmaps to address issues such as the trial signup and demo process. As a result, we observed a 25% increase in conversions within three months after companies participated in the trials.
Next Steps and What We've Learned
After the study, we created journey maps and personas for individuals navigating the Cloud Pak for Business Automation steps. This highlighted significant flaws in the overall experience, prompting the team to develop roadmaps to address issues such as the trial signup and demo process. As a result, we observed a 25% increase in conversions within three months after companies participated in the trials.
