Threaded comments

Background

Client Vision Critical

When I joined the team, they had been maintaining a new product, which was introduced to the business through acquisition. The team hadn’t introduced any new functionality, but instead, were maintaining the product with technical debt items. The team was looking for a quick win and leadership wanted us to focus on delivering something valuable for our first project.

My key responsibilities were:

• Understanding and scoping product requirements.

• Creating wireframes and high fidelity prototypes for a responsive web platform.

• Setting and evaluating success metrics of the final product.

Problem and Objective

User problem

Brands engage with customers by making blog-style posts. Members interact with posts by reading, liking or commenting. Comments appeared below posts in reverse chronological order.

Problems identified with this approach were:

• For members, jumping into a conversation was a disjointed experience. Addressing an earlier comment would post a reply at the bottom of the comment list, negatively affecting readability.

• For administrators, it was difficult to collect feedback and probe further into member responses.

After reviewing and prioritizing our backlog, we chose to tackle this problem for several reasons:

• We did not have time for any up front discovery work and needed something we could execute asap.

• Highly requested by internal stakeholders and customers.

• We could fit this into a manageable scope.

• Technical effort had relatively more known variables.

• Impacted all of our users as opposed to just a subset of users.

Objective: Allow community members and administrators to respond to comments in context and improve readability of conversations.

Scoping

Analyzing other implementations

As part of my research, I investigated the commenting functionality of our indirect competition to analyze design patterns. To properly scope the requirements I would need to understand the different patterns that can form a comment/reply model. Understanding design patterns helped us decide what to leave off the table for the initial version.

Scoping is about creating the sandbox within which the UX designer will work, defining the problem that is being solved, and the measures of success. Clear scope upfront will lead to better outcomes at the end of the work.

- Article

Competitive research would also inform my design process by allowing me to brainstorm new solutions, reuse common interactions, and predict potential pitfalls.

Multiple levels of replies

Replying to a reply can promote deeper conversations. However, more complexity is introduced to the experience. Deep commenting threads can cause users to get lost in a conversation or distracted. Rolling back this design decision would also be non-trivial. For these reasons, we chose to only have one level of replies.

Expanding replies

We predicted most users would primarily scan comments before deciding to go further into threads of interest. Collapsing and expanding replies would allow users to probe further as desired, optimizing the experience of scanning through comments. Initially, we decided to leave expanding replies off the table because we weren’t predicting long (>10) threads*.

*Upon review, we backtracked and decided to add this functionality because we overestimated the effort to implement.

The composer

Default Composer functionality

• URLs: Adding a URL opened a modal window and applied a large, but appealing visual style.

• Images: Image uploads were unlimited and styled to fit the width of a post. Users also had the option to add images as a scrolling slide slideshow.

• Reorder: Text and other content could be reordered with drag and drop functionality.

• Headers: Text could be treated with HTML header tags to provide visual hierarchy and improve Search Engine Optimization (SEO).

• Documents: Documents and videos could be attached or embedded providing an interactive experience.

I held a meeting to make decisions on the functionality we would keep, remove, or modify for our use case.

Designing a solution

Solutions

Using the scoping discussions as a guide, I brainstormed and mocked up different solutions to each problem or user task.

Adding a URL

I chose to remove the default embedded URL styling and kept the design simple with link-styled text. I also created mocked up and shared both designs to illustrate the added complexity.

Images

I chose to display images side by instead of using a slideshow/carousel alternative. Clicking or tapping the images would open up a lightbox (larger image). This decision was made to optimize scanning and minimize interactions on mobile screens.

Tagging

Tagging other members (typing ‘@” before the name) sent them a notification, prompting a visit to the conversation. This worked great already but needed to be optimized for small screens.

Measuring the outcome

Our organization had a process of releasing features to a closed group of users through a beta program. The beta program lets customers opt-in to receiving the product feature outside of our scheduled quarterly release cycle.

Awareness

Measuring user awareness of the problem helps identify if it is worth solving. Sign-ups for the beta program suggested that many users were aware of the problem and interested in a solution. This also gave promising signs for future adoption of the feature.

The 27 customer sign-ups were:

• The highest number of sign-ups from that quarter’s releases.

• The highest number of sign-ups that the team has ever received.

Adoption

During the beta program, we were able to track how many of the 27 customers used the feature. We used qualitative methods by monitoring incoming bugs, reaching out to customers etc. We were able to estimate that a maximum of 11 communities did not use the feature. In retrospect, we could have made different decisions to instrument the product and track the number exactly.

Engagement and measuring success

Three months into the beta program we were able to see if we moved the needle on our success metric.

• Target change: +10% total comments

• 90 day difference: +3% total comments

We made the decision to measure again after 6 months with a larger sample of activity. The feature was still deemed successful at this stage and was rolled out to all 300+ member hubs.

Post Mortem

Looking back on projects retrospectively allows me to take the opportunity to reflect and better my craft. There are a few things that I would do differently with this particular design challenge if given the opportunity.

Planning for consistency

We managed to build a brand new system to reply to comments. While I think the design was successful in solving the problem at hand, we introduced new design patterns. Posting an original comment (that is not a reply) still used the Composer tool. Looking holistically at the product, there are some parts of the reply functionality that we could integrate into the commenting workflow. I would champion these changes and frame the changes as another experiment to further drive our engagement metrics forward.

Scoping

Our scoping work had been one of the highlights of this design process. However, during delivery, we took too many liberties with our original scope and ended up bring in additional solutions which improved the functionality but prolonged development. I could have tried using different artifacts to visualize future releases so the team could see how the functionality evolves, while not getting too far ahead of ourselves trying to implement everything at once.

Metrics

I am happy with our choice of success metric. We wanted to see total comments increase, so we instrumented the reply functionality to be logged as a comment in our database. The flaw with this approach is that we weren’t able to track replies separately from comments. We should have added another entry to track replies. This would help us measure the influence of the new feature on the success metric as well as to measure how different communities are adopting the new functionality.