Compounding Thoughts

Share this post

Building in Public: How to determine what features to build with limited information

compoundingthoughts.substack.com

Discover more from Compounding Thoughts

A newsletter on all things consumer, commerce and culture.
Continue reading
Sign in

Building in Public: How to determine what features to build with limited information

Making decisions with imperfect information, competitor usage analysis, knowing when to fold, deserving success

Sam Blumenthal
Jul 19, 2023
Share this post

Building in Public: How to determine what features to build with limited information

compoundingthoughts.substack.com
Share

Like most things in life, being a startup founder is about tradeoffs. With highly constrained resources — specifically time, capital and engineering resources — one of the my most critical responsibilities is resource allocation.

Yet, this presents a conundrum. At the moment when resource allocation is most constrained, the information available to one to make a decision is most limited. In many ways, it’s like trying to win the World Series of Poker with a fraction of the chips and only knowing a single card in your hand.

Thanks for reading Compounding Thoughts! Subscribe for free to receive new posts and support my work.

Hence, the high failure rate. It’s hard to win that game because the game is not designed to be won that way.

Thus, it becomes the job of the CEO and founder to either gain more chips (acquiring more resources) or figure out how to play your hand. Yet, for all but a select few, it’s hard to get more chips without proving you know how to play the game and win some hands.

So what do you do?

First, you must understand the dynamics of the game you are playing and what information is (1) accessible and (2) valuable.

In Texas Hold Em, it would be valuable to know what other cards your opponents have, but that information is not accessible. Conversely, it is quite accessible to know the community cards dealt during the flop (the first three cards played by the dealer face up), but if you don’t know that a straight beats a two pair you might fold a winning hand.

Yet, there is more nuance here. Information has transactions costs. User research, for example, is sales. It requires hours prospecting, outreaching and then, hopefully, interviewing. It also requires preparation to understand what information from a given user research sprint is valuable — as refined in specific hypotheses — and what questions and interview patterns unearth that information with the least amount of bias.

Over the past month, I conducted somewhere between 50+ user research interviews for my startup. As we are undergoing the natural pivot, I will keep the user segment and nature of these interviews private. But they consisted of three specific personas that required their own specific outreach strategies. The easiest segment allowed me to go direct. The second hardest segment required me to leverage Instagram’s social graphs. The hardest segment required me to scrape the Instagram like history of supply in a certain vertical then manually review 5,000+ profiles to compile follower information and potential email contacts. From there, I conducted hundreds of cold email outreach with more emails ignored than not. Alas, transaction costs.

So, when transaction costs are high and information and resources are constrained, grunt work and creativity become extremely valuable skillsets. Creativity is an important attribute as it suggest a propensity to think-outside-the-box. This helps identify unconventional solutions to a given problem: in this case possible channels and resources from which one could procure valuable information that is highly accessible.

The following analysis is one such example.

I am sharing this work to provide documentation of my decision-making process as a founder and leader for future investors and employees. I also hope that other entrepreneurs and would-be-founders find this process as a call to action to embrace the grunt work and get creative when making decision with imperfect information.

This analysis, combined with user research and certain MVP test data, gave us the conviction to fold a hand at the moment that the bet was raised. This, in hindset, was a good decision. More importantly, our decision making process was sound.

The role of a startup leader is not simply resource allocation. Communication is vital. How you communicate internally how certain decisions were made and why is critical to creating alignment.

I find writing as the most effective tool for not just synthesizing imperfect information to arrive to a decision, but also as a mechanism to explain how we arrived their and why it matters.

Boats move faster when the crew paddles in the same direction. In the high seas of startups, the water can get choppy and the wind fierce. When communication is difficult, write.

Overview

This analysis seeks to understand how different products features are used by community users. Specifically, we will explore the relative value creation of different product features by measuring post frequency by feature. We will also measure the value that certain features create for the community as mediums of interaction based on the average likes per post and comments per post by feature type. We will focus on three communities that have proven product-market fit in a category of interest (beauty), they are:

  • Glossier (Reddit)

  • Jones Road Beauty (Facebook)

  • Beauty Pie (Facebook)

And one community (The Daily Planner) from Mighty Networks that lacks the scale of the above communities but has signs of durable engagement.

Research Question

What features create the most value across top performing communities and are there areas where users needs are being undermet?

Methodology

This analysis seeks to answer our research question across four independent sample sizes of 100 posts from the four referenced community. Jones Road Beauty, for example, had 663 posts over in the last month. We believe the number of posts over a 30 day period is the right population size as it is exhaustive enough to capture all community use cases and the usage of product features to best meet them. Our sample size of 100 implies a 95% confidence interval with a 9% margin of error.

For each post, we recorded the type of product feature used, the number of likes and the number of comments.

  • For Glossier’s Reddit Community we also recorded the post’s flair: Reddit’s native theme tagging feature.

  • For Jones Road Beauty’s Facebook Community, we also recorded the post creator’s ‘badge’, the emotion expressed by the post and the post content type by use case.

  • For Beauty Pie’s Facebook Community, we recorded the post creator’s ‘badge’ and post content type by use case.

Goal

The goal of this analysis is to evaluate which features are currently being used in existing communities, especially in context of (1) the specific use case and (2) what other features are available.

This will allow us to better understand what existing (or non-existing) features met user needs for certain use cases and expressions. By doing so, we can determine, prioritize and potentially kill certain elements of our roadmap.

Analysis

Reddit

Reddit provides a relatively wide feature set for their content. These include:

  • Text-only Posts

  • Image-focused Posts

  • Videos

  • AMAs

  • Polls

  • Discussion Boards / Moderator Threads

  • Brand Announcements

Glossier

In Glossier’s 42k member Reddit community, users solely opted for Text-only posts and Image-focused posts form factors across the various use cases employed by the community. Image-focused posts had nearly an equivalent usage as Text-only posts. Interestingly, users did not employ Polls in use cases where one might imagine it to be an effective feature. A single post employed a Moderator Thread / Discussion Board.

Source: Compounding Thoughts, Banter

Moreover, Image-focused posts had 5.8x greater average like per post and nearly the same number of comments as Text-only posts. This implies that this feature (well rendered images with a header) is both a popular medium for creators to express themselves and creates more value to the community as they reward the creator with upvotes and provide them with the requested feedback or support outlined in the post via comments.

Source: Compounding Thoughts, Banter

Additionally, all posts in Glossier’s Reddit community require the creator to include a “flair” which adds as a “tag” to denote the topic of the post. The Flairs are:

  • Collection/haul

  • Discussion

  • Questions

  • Issues

  • Tips

  • News

  • Other

  • Review

  • Swatches

Furthermore, we analyzed the feature frequency by flair. By doing so, we can evaluate and rank community use cases and understand which product feature best meets the Job to be Done of the user to express a certain use case.

Source: Compounding Thoughts, Banter

Similarly, we analyzed average likes and comments per post across different flairs. This allowed us to observe which feature provided more relative value to the community in different use cases.

Some use cases, such as Questions, are more frequently expressed in text (86% of the time). But in the circumstances that they are expressed in image-focused posts, it creates significantly more value for the community who compensate the creator with 12x more likes and 2.4x more comments on average. Interestingly, community members leveraged image-focused posts 42% of the time for Discussions — the most common use case found in the community. When they did so, they created significantly more value on average for the community (6.2x more likes) and fostered slightly more responses on average, which ultimately is the goal of a ‘discussion’.

Source: Compounding Thoughts, Banter

Actionable Insight

Users do not value — or do not know that they value* — polls as a feature to express the different use cases across community.

*This may be a function of product design and UI.

*If further research disproves this conclusion and informs that users find polls valuable, then the following lessons from Reddit’s UI should be severely weighted in any UI design.

Reddit’s posting UI offers three features at the top of the feed for users to quickly employ to create a post: text, image and link. As we have seen, this UI has a direct impact on which features users’ employ.

Source: Compounding Thoughts, Banter, Reddit

Upon clicking any of those options, a user is sent to a new view where they complete the remaining actions for the specific feature and select a relevant flair. It is in this view that a user now sees the option to create a poll. This creates added friction and confusion.

Source: Compounding Thoughts, Banter, Reddit

Images create more value for the community and are used across many use cases; we should attempt to make image-focused posts the default

Our product design should make the act of uploading or posting an image extremely easy and the default action should permit sharing image-focused posts as easy as sharing text-only posts. This means the original call to action to create content should include an action relating to sharing images.


Moving on, both Jones Road Beauty and Beauty Pie host their communities in Faceook Groups.

Facebook Groups

Facebook Groups credentials users with badges. Facebook’s badge system is determined based on the accumulation of Contributions. Users can have multiple badges.

Source: Compounding Thoughts, Banter

Facebook FAQs

What are Contributions?

Contributions reward you for being a great contributor in your group! As you participate in your group, you’ll be eligible to earn different badges.

What’s the value of holding a badge?

When you get a badge, others will be able to recognize you as a top group contributor! It shows you regularly contribute to the group by creating posts or leaving comments.

Do these carry over to my other groups?

No, each of your badges are specific to one group.

Facebook Groups also provide a wide range of features. These include:

  • Text-only Posts

  • Image-focused Posts

  • Featured Posts

  • Polls

  • Videos

  • Rooms

  • Feeling/activity

  • Check in

  • Create Event

  • Tag event

  • Tag people

  • Livestreams

  • Reshares

  • Embedded Media

Nonetheless, users have willed certain “features” into existence with their own makeshift versions that are a clunky repurposing of links, photos and text. These include:

Clunky Product Tagged Photos

Source: Facebook

Shoppable Product Launches

Source: Facebook

Expert Consultation

Source: Facebook

Product development Input and Feedback

Source: Facebook

Direct Messaging the Brand

Source: Facebook

External Discounts

Source: Facebook

Gifting/Resale

Source: Facebook

Competition/Challenge

Source: Facebook

Shoutouts

Source: Facebook

Screenshot (PDP Image, IG Post)

Source: Facebook

Jones Road Beauty

Jones Road Beauty’s 18.2k member Facebook Community used a range of product features with text-only being the most common. Importantly, user-created “clunky Tagged Product Photos” was the second most common feature used for content creation. Clearly, users gained value from sharing a photo — most commonly a selfie of themselves — and ‘tagging’ the different products that they were wearing. But given the lack of tooling for this feature, users did so in plain text by ‘tagging’ the products in the post’s caption.

Again, polls are not being employed as a feature across all the different use cases for content creation in the community. Moreover, Facebook’s UX is designed to push users to create ‘Rooms’. However, this product is seldom used and does not appear to create any value for the community.

Source: Compounding Thoughts, Banter

As we saw with Glossier’s Reddit Community, image-focused posts appear to create more value for the community. Image-Focused Posts had 5.5x more likes and Clunky Product Tagged Photos had 4.7x more likes than text-only posts. Moreover, Imaged-focused Posts had 1.7x more comments and Clunky Product Tagged Photos had 1.5x more comments than Text-only Posts. This implies that this feature creates more value to the community as a means of consumption (rewarding the creator with likes) and social interaction/engagement (rewarding the creator by spending the time and energy to provide them with the requested response).

Source: Compounding Thoughts, Banter

Moving on, creators of posts in this community ranged in all types of engagement, which was measured by their badges. The most active contributor were individuals with the Top Contributor badge, followed by New Members and users with no badge.

Source: Compounding Thoughts, Banter

While Top Contributors, New Members and users with no Badge post the most in JRB community, they create the least amount of value for the community on a per post basis as measured by likes per post and comments per post. These users use text-only post features the most frequently.

Source: Compounding Thoughts, Banter

Admin, Moderators, Group Experts, Visual Storytellers and Rising Stars create a lot of value for the community but do not post frequently. These users use text-only post features the least frequently. Instead, specifically in the case of Admins and Visual Storytellers, they often employ ‘user-created’ product features like Clunky Product Tagged Photos. These posts, in addition to Embedded External Media and Reshared Posts, appear to create more value for the community than by which they are produced.

Source: Compounding Thoughts, Banter

What emotions were users most commonly expressing when sharing posts and interacting with the community?

Users most commonly created posts in the community and sought out interaction to share their gratitude of JRB and the happiness and contentment it has given them in their life. Often, they were curious about products and what to buy given certain use cases and conditions, as well as what the community thought or recommended regarding how they can improve their Job to Be Done.

Source: Compounding Thoughts, Banter

The community most valued posts where users shared vulnerability, excitement, confidence, happiness, enthusiasm and gratitude. The community less valued posts that expressed uncertainty, frustration and curiosity, although they provided support in the form of comments.

Source: Compounding Thoughts, Banter

Negative emotion posts were most commonly shared via text-only product posts whereas the valued emotional sentiment outlined above leveraged imaged-based posts, specifically Clunky Product Tag Photos.

Source: Compounding Thoughts, Banter

We see a similar distribution of usage across the different use cases and content types, which we discusses in Online Communities: What Makes Them Tick?.

Source: Compounding Thoughts, Banter

Actionable Insights

  • Product Tagged Photos appear to be the most actionable feature to build.

  • As we saw with the other communities, image-focused posts appear to create more value for the community. This implies that this feature creates more value to the community as a means of consumption (rewarding the creator with likes) and social interaction/engagement (rewarding the creator by spending the time and energy to provide them with the requested response).

  • Reshared Posts, Embedded External Media, Shoppable Product Launches and Videos create a lot of value and engagement from the community but are not used frequently by post creators as a means to express certain community use cases.

  • Polls are never used.

  • While Facebook’s UX pushes users to create Rooms, it does not appear to have product-market-fit with the use cases of the community and is rarely used.

  • Top Contributors, New Members and users with no Badge post the most in JRB community but create the least amount of value for the community on a per post basis as measured by likes per post and comments per post

  • Admin, Moderators, Group Experts Visual Storytellers and Rising Stars create a lot of value for the community but do not post frequently. Product features and UI should be designed to get these users to post more. One way to do so is by investing in the feature sets that they are duct-taping together.

    Some potential features that could best serve these users include:

    • Shoutouts

    • Reshares

    • Embedded Media

    • Direct Messaging

    • Livestream

    • Integrated Shoppable Product Launches

    • Expert Tooling

    • Color Recommendation and Selife Submissions

      Source; Facebook
    • Tickets and events

    • Product Development Input and Feedback Requests

  • Conversation Starters post at a high frequency but appear to be creating less relative value for the community and not spurring as much conversation as other user types (as measured by average number of comments per post).

  • Embedded External Media and Reshared Posts appear to create more value for the community than by which they are produced.

  • The JRB community most valued posts where users shared vulnerability, excitement, confidence, happiness, enthusiasm and gratitude.

  • Negative emotion posts were most commonly shared via text-only product posts whereas the valued emotional sentiment outlined above leveraged imaged-based posts, specifically Clunky Product Tag Photos.

  • Facebook’s Posting UI pushes Rooms, People Tags, Check Ins and Feeling/activity — product features that are never used and are more suited for general Facebook usage not community usage. Similar to the issue with Reddit, users do not find value in polls — or do not know that they find value in polls given how buried they are in the user journey to create a post.

Source: Facebook

As a result, polls are not being used in obvious use cases.

Source: Facebook

Beauty Pie

Beauty Pie’s 24.4k member Facebook Community used a range of product features with text-only being the most common. Compared to JRB, users create less “Clunky Tagged Product Photos”, but instead use images in other use cases. BP users have created their own feature in the form of ‘screenshotting’ to best meet a specific use case relating to getting information at specific points in the customer journey while they shop on site.

Again, polls are not being employed as a feature across all the different use cases for content creation in the community. Moreover, Facebook’s UX is designed to push users to create ‘Rooms’. However, this product is seldom used and does not appear to create any value for the community.

Source: Compounding Thoughts, Banter

As we saw with the other communities, image-focused posts appear to create more value for the community.

Compared to JRB, users create less “Clunky Tagged Product Photos”, but instead use images in other use cases. BP users have created their own feature in the form of ‘screenshotting’ to best meet a specific use case relating to getting information at specific points in the customer journey while they shop on site.

Source: Compounding Thoughts, Banter

Moving on, Users with no Badge post the the vast amount of content in the BP community, followed by Conversation Starters, New Members and Visual Storytellers. This community has much less diverse user base by contribution and do not have many power users actively contributing.

Source: Compounding Thoughts, Banter

Visual storytellers and users with No Badge create the most value to the community as measured by both likes and comments.

Source: Compounding Thoughts, Banter

Image-based product such as tagged photos, image-focused posts and screenshots create the most value for the community and are most commonly used by user types that create the most value for the community.

Source: Compounding Thoughts, Banter

Actionable Insights

  • Image-based product such as tagged photos, image-focused posts, screenshots create the most value for the community and are most commonly used by user types that create the most value for the community.

  • Compared to JRB, users create less “Clunky Tagged Product Photos”, but instead use images in other use cases. BP users have created their own feature in the form of ‘screenshotting’ to best meet a specific use case relating to getting information at specific points in the customer journey while they shop on site.

  • Users with no Badge post the the vast amount of content in the BP community, followed by Conversation Starters, New Members and Visual Storytellers.

  • The Community suffers from the lack of posts from Admin, Moderators and Group Experts, who create a lot of value for community members.

  • Admins ‘user-create’ brand announcements. These posts created the most value to the community and saw strong comment contribution. However, the lack of feature set for this use case seems to have created friction that likely resulted in less content creation and community value as a result. Features focused on getting Admins, Moderators, Group Experts and Rising Stars should be prioritized.

  • Beauty Pie Community members currently take a snapshot of the PDP and then share that image to the Facebook community to get feedback because that’s where individuals they trust reside. This creates friction and takes the user away from the store. Nonetheless, this use case and hacked together feature drives conversation in the form of comments. This feature is used to express a number of use cases in different parts of the customer journey.

How do shoppers leverage brand communities during the customer journey?

Interactions on PDP

Product Feedback to help inform a buying decision even when reviews are available

  • Requesting specific feedback from community regarding product X given condition/question Q

Seeking information and support for a specific problem

  • Requesting support and advice from the community regarding problem P that user is facing with product X

Seeking information and advice from the community on how to better use a product

  • Requesting information from the community regarding how to effectively use product X given condition Q in order to improve its performance for use case Z

Product review

  • Sharing feedback with the community regarding how product X has performed on use case Z given condition Q or comparison to product Y

Complaint

  • Sharing a complaint with the community regarding how issue S has made product X underperform for use case Z

By eliminating the need to screenshot and instead creating a feature that enables the creator to reference a certain product and distribute natively alongside the customer journey, we can increase the value that these posts provide for the creator by making them more convenient.

Interactions at Checkout

Three questions as I “load up my JR cart” for travel friendly items

Product Comparison for specific use case

  • Requesting comparative feedback from the community regarding how product X (in cart) compares to similar product Y for use case Z

Cleansing stick (X) - do you like it equal to or better than micellar water (Y) for makeup removal (Z)?

Product Feedback for specific use case given certain constraints

  • Requesting specific feedback from the community regarding how product X (in cart) performs for use case Z given condition Q

Oil stick (X) - has anyone with an oily Or combo complexion (Q) tried it on their roscea or flaky spots (Z). Don’t want to make pours worse but always looking for things to help

Product Variation Feedback for specific use case

  • Requesting specific feedback regarding specific variations of product X (X1, X2, X3, etc.) for use case Z given condition Q

Shimmer oil - Midas (X1) or Bronze (X2) for a redish face (Z)? I love (loooove) golden hour MB and use bronze MB for a little contour. But I’m confident I will not be able to handle MB with my natural oils and humidity (Q). Already have opal SFO and loved it in winter.

Input on how product can improve

  • Providing specific input regarding how product X can be improved and how the brand can better address use case Z given condition Q

Input for Sydney… it would be interesting if there could be a lighter weight version of WTF (X), like the light weight cream vs miracle cream. Moving into summer Id like the coverage that WTF does provide in smoothing about my color (Z), but don’t need the extra moisture (Q).

Mighty Networks

Mighty Networks provides a wide range of features. These include:

  • Text-only Posts

  • Image-focused Posts

  • Featured Posts

  • Polls

  • Hot and Cold

  • Questions

  • GIFs

  • Introductions

  • Embedded Media

  • Videos

  • Rooms

  • Full-screen Image

Nonetheless, like we have seen elsewhere, communities leveraging MN have willed certain “features” into existence with their own makeshift versions that are a clunky repurposing of links, photos and text. These include:

  • Clunky Product Launch

  • Clunky Product Tag Photos

  • Challenges

  • Product Input

  • Screen Recording

  • Gifting/Selling

  • Brand Announcements

The Daily Planner is a community that recently transitioned from Peerboard to Mighty Networks.

This community used a range of product features with Image-focused Posts being the most common. Interestingly, text-only posts were quite rarely used in comparison to other communities. The third most common product feature used was questions, which was often used instead of polls to gather feedback and information from the community as part of a users Job to be Done.

Similar to to other communities, we saw users will certain features into existence, specifically clunky product launches that tried to embed commerce and shopping instances within the community, as well as clunky product tagged photos and challenges.

Source: Compounding Thoughts, Banter

Again, polls and hot cold are infrequently employed as a feature across all the different use cases for content creation in the community.

As we saw with the other communities, image-focused posts appear to create more value for the community. Gifs and embedded external media often conveyed emotion and lightheartedness that entertained and please the community which saw the most amount of likes. Users also gained value from posts requesting and providing product input (not using the polls feature) and found value in the common connection involved in challenges.

Source: Compounding Thoughts, Banter

Questions by and far received the most median responses from the community. Image-focused posts, GIFs and product input use cases saw high engagement in terms of the community commenting on posts that employ these features.

Source: Compounding Thoughts, Banter

The Admin or Host was an extremely active contributor in the community and saw the most likes and comments per post.

The Admin/Host predominantly employed the features that created the most value for the community, including:

  • Embedded External Media

  • GIFs

  • Product Input

  • Challenges

Interestingly, Questions were used across user types.

Source: Compounding Thoughts, Banter

Features for Admins

Aggregating our analysis, we can see the frequency by which these features were used by Admins across the different communities.

Source: Compounding Thoughts, Banter

What we did with this analysis, unfortunately dear reader, I am keeping close to the vest.

Closing Thoughts

Being a founder, so I have learned and come to love, is about doing the work. It’s about trusting the process, as my friend Sam would say. As my team knows, and often hears me say, production makes proficiency. I wrote about this a bit in What Makes IP Valuable. What this means is that you don’t become Michael Jordan without first putting in the hours practicing, training, working out, failing, falling, hurting, and what not. That work — that production — is worthless to everyone but him, because he gains value from the process of getting better. 99% of your time as an entrepreneur is spent doing work that’s main source of value is internalized not externalized.

Everyone has a will to win but very few have a will to prepare to win.

- Vince Lombardi

One of our values at Banter is Hone Your Craft. You hone your craft when know one is watching, working on things that no one will pay for, arriving at results that nobody ever sees. But that’s how you compound a skillset and a will to prepare to win.

This, perhaps more than anything else, is the most important trait that we look for in potential hires. A relatively controversial question that I frequently ask in interviews is, “why do you deserve to be successful?”.

It is blunt, but I find it effective. All humans want, and so wanting to be successful is an extremely easy thing to say and does not psychological threaten our sense of integrity.

Deserving, I have found, is an interesting filter. For one to say they deserve something, it implies that they have already born some cost and accomplished something worthy — whether a quality or an achievement — to arrive at that moment in time. When we say we deserve something, it intuitively challenges our integrity because it is a function of where we have been rather than where we hope to go. We are green when we want, we are hardened when we deserve. It is much harder to fake and even more importantly, vibrantly shines through when genuine.

Why do you deserve to be successful?

Thanks for reading Compounding Thoughts! Subscribe for free to receive new posts and support my work.

Share this post

Building in Public: How to determine what features to build with limited information

compoundingthoughts.substack.com
Share
Previous
Next
Comments
Top
New
Community

No posts

Ready for more?

© 2023 Sam Blumenthal
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing