Mixpanel makes it easy to start collecting events and building dashboards. The harder part is knowing whether those dashboards represent real product usage.
A report can look polished while counting employees, automated browser tests, web scrapers, duplicate events, or actions that users attempted but never completed. It can also be technically correct while answering a different question than the one its creator intended.
Over time, I have learned to define terms, filter out noise, and validate the data before building boards or relying on metrics. These examples use Mixpanel, but most of the principles apply to any event-based analytics provider.
Start with Business Goals and Definitions.
Before creating events or reports, I write down the business goals and questions I am trying to answer. I add them to the Mixpanel board so each report stays connected to a goal.
Examples
- What action shows that a user experienced the product’s core value?
- What percentage of new users reaches that action?
- Which early behaviors are associated with users returning?
- Are product changes improving activation or retention?
- What do users do after sending a message?
- How many users reach a particular page on web and native?
- How long do users stay on the site after reaching that page?
Time on site can also expose bot traffic. If zero-to-two-second visits make up the largest group, that traffic deserves a closer look.
Those questions determine which events and properties I need. They also reveal when a report is answering something else. To stay organized, I keep a glossary that links each question to the report that answers it.
Mixpanel recommends creating a tracking plan that connects business goals to user flows, events, and properties. You can also put definitions on the board so people do not have to guess what each report counts.
Terms Worth Defining
- User
- Unique user
- Active user
- Session started
- Page visitor
- Value moment
- Retained user
A useful definition includes the events being counted, required property values, time window, platform coverage, and filters. It also explains what the metric excludes and what conclusions it cannot support.
For example, a value-moment metric may show that someone used a meaningful feature without proving they completed their task or returned later. A page-visitor metric may include anonymous traffic without showing whether those visitors signed in. A session metric may count access without saying anything about engagement.
Putting those limits beside each report prevents a convenient metric name from making a larger claim than the data supports.
Decide Who Should Count.
Every metric definition should also say who is included.
Employees, developers, QA testers, and service accounts can generate a surprising amount of activity. A feature may look healthier simply because the team building it uses it frequently.
Mixpanel provides several ways to exclude internal and unwanted traffic, including checks for known email domains, IP ranges, and service accounts. Its reserved $ignore property drops marked events during ingestion.
Our team uses cohorts to define reusable groups based on shared properties or behavior. I might define active users as authenticated accounts that completed a core product action within the past seven days, then apply that cohort across reports.
An active-user cohort can reduce some bot noise, but it will not identify every bot. Bots may perform core actions, and legitimate automated tests may use headless browsers. I filter known automation separately using properties such as user agent, environment, test account, or service identity.
A coworker recently pointed out that browsers expose navigator.webdriver when automation controls them. We now send that value to Mixpanel as is_automated_browser. It helps us filter browser tests and other automation without pretending it catches every bot.
I define identity rules at the same time. Mixpanel uses distinct IDs to determine uniqueness and follow users through funnels, sessions, and devices. Poor identity management can make one person appear as multiple users or create artificial funnel drop-offs.
Establish Boring Baselines.
Before measuring new features, I keep a small group of stable baseline metrics:
- Unique page visitors
- Successful logins
- Unique authenticated users
- Sessions containing meaningful activity
- Users completing the product’s core action
A core action that most genuine users perform gives me a behavioral reference point for page traffic, experiments, feature adoption, and suspicious changes in usage.
I also document whether each baseline counts events, sessions, or unique users. Unique-user counts are not additive. If the same person is active on three days, adding the three daily totals counts that person three times. A seven-day unique-user total needs its own seven-day query.
Make the Metrics Part of the Routine.
The Mixpanel MCP server lets an agent explore events, properties, funnels, retention, cohorts, and data-quality issues. I use it to ask follow-up questions, investigate unexpected results, and pull data when deciding whether to keep polishing a feature or remove it. Then I bring that data into Slack conversations, PR descriptions, and stakeholder meetings.
I have also been experimenting with an agent that retrieves a small set of stable metrics and posts them daily in a team channel. Each update includes weekly averages, the date range, metric definitions, the cohorts used, and a link to the report. The goal is to make trends easy for the team to interpret. Ideally, updates start a conversation, but I have not cracked that part yet.
Use Imperfect Data Honestly.
Analytics data will always have gaps, but stable trends can still guide product decisions. That requires keeping definitions, filters, and collection methods consistent.
An instrumentation change, identity fix, new platform, or bot filter can create a shift that looks like product behavior. I document those changes and share them with the team before we interpret the trends.
If I were improving a Mixpanel project today, I would define who counts, create the necessary cohorts and filters, and validate one complete user flow. Only then would I start treating the dashboard as something worth watching.