Why Your Phone Bill Is Genuinely Hard to Build

Every call you make on a mobile network generates a record. Every megabyte you download, every text you send, every roaming session, these are events in a network that has been engineered to capture them. What happens to those records between the moment they're captured in the network and the moment they appear on your monthly bill is the telecom billing stack, and it's one of the more technically interesting infrastructure problems in enterprise software.

I spent eighteen months working on billing software for a mid-size mobile operator. Before that I had a rough understanding of what billing systems do, "they calculate your bill", and no understanding of how they do it. Here is what the stack actually looks like.

The Network Generates Data Constantly

A mobile network's core elements, the MSC (Mobile Switching Centre) for voice calls, the SGSN/GGSN or P-GW for data sessions, the SMSC for SMS, generate call detail records (CDRs) for every event. A CDR is a structured record containing: the calling and called numbers, the start time and duration or data volume, the network element that generated the record, the tariff class applied, and various technical identifiers. At an operator with several million subscribers, the volume of CDRs generated per day runs into the hundreds of millions.

These CDRs are generated by multiple network elements in potentially different formats, each vendor's equipment produces slightly different CDR formats with different field names, different encodings, different timestamp formats. A network that has upgraded its equipment over many years may have several generations of hardware running simultaneously, each producing CDRs in the format used at the time of installation.

Mediation: The Normalisation Layer

Between the network elements and the billing system sits the mediation layer. Its job is to collect CDRs from every network element, normalise them into a single consistent format, deduplicate any records that were generated more than once (which happens when network elements retry record delivery after a transient failure), enrich them with additional data from external sources (like the subscriber's current plan and account identifier), and make them available to the billing system in a reliable, ordered stream.

Deduplication sounds straightforward until you encounter the reality: the same call might generate a CDR from both the originating MSC and the terminating MSC. One of those should be billed; the other should not. The deduplication logic has to identify that these two records represent the same event, determine which one is the canonical record for billing purposes, and suppress the other. Getting this wrong in either direction is expensive: a missed CDR means an unbilled event, which is lost revenue; a duplicate CDR means a customer is billed twice, which is a billing dispute.

Mediation systems process CDRs in near real time, but the billing itself is typically periodic, daily or monthly. The mediation layer has to handle the case where a CDR arrives late, sometimes days late, because a network element experienced a failure and only restored its stored records after reconnecting, and determine whether to include it in the current billing period or in a next-period correction. Most operators have a configurable late-CDR window: records arriving within N days of the event date are included in the current bill; records arriving later generate an adjustment in the next period.

Rating: Turning Usage into Money

After mediation, CDRs are passed to the rating engine, which applies the subscriber's tariff to calculate the charge for each event. The tariff might be: the first 500 minutes of calls per month are included in the plan, additional minutes are charged at 1p each, international calls to Zone A countries are charged at 15p per minute regardless of the plan allowance, roaming data is charged at 0.002p per kilobyte.

Rating is where the tariff structure's complexity becomes an engineering problem. Modern mobile tariffs include: monthly plan allowances (voice, data, SMS) that carry over or expire depending on the product; bolt-on packages that modify the base tariff for specific use cases (a roaming add-on for international travel, a data speed-up for heavy users); promotional rates for new customers that apply for a defined period; family plan shared allowances where multiple subscribers share a pool; loyalty benefits that modify rates after N months on a plan.

The rating engine evaluates each CDR against the subscriber's current entitlements, their plan, their bolt-ons, their current cycle's remaining allowances, and produces a rated event: this CDR costs this amount, drawn from this allowance bucket. The cumulative rated events for a billing cycle, when aggregated across all subscribers, become the invoices.

Invoicing and the Bill Run

At the end of the billing cycle, the bill run aggregates all rated events for each subscriber, applies any credits or adjustments, adds the monthly plan charge and any one-off fees, calculates the final invoice amount, and produces the invoice in a format suitable for delivery to the customer and for posting to the accounts receivable system.

The bill run is a batch process that, at a large operator, processes millions of invoices within a defined window. The window is constrained by the downstream systems: the printing and mailing service needs the files by a certain time, the bank debit instructions for direct debit customers need to be submitted by a certain date, the accounts receivable posting needs to happen before the accounting period closes. These deadlines are fixed and non-negotiable, which means the bill run has to complete reliably within its window regardless of upstream problems.

Failure modes in the bill run, late CDRs that arrive after the run started, rating engine errors that produce negative charges, missing plan records for subscribers who changed plans mid-cycle, have to be handled gracefully: either included in the current run with the best available data, flagged for manual correction, or queued for a supplementary run. An invoice that is generated incorrectly creates a billing dispute. A billing dispute is a customer service interaction, a potential churn event, and a revenue recovery cost. The quality of the bill run directly affects customer satisfaction and operational cost in a way that most billing software teams feel very concretely.

Revenue Assurance

Revenue assurance is the process of checking that all billable events have actually been billed correctly. In a system that processes hundreds of millions of events per day, statistical leakage is expected, some CDRs will be lost, some will be duplicated, some will be rated incorrectly because of a tariff configuration error. Revenue assurance is the function that catches these issues before they compound.

A revenue assurance team typically runs continuous comparisons between the raw CDR volume from the network and the rated CDR volume in the billing system. A systematic gap, say, 0.1% of CDRs from a particular network element are not appearing in the billing system, indicates a mediation problem. A systematic pricing anomaly, a particular call type is being rated at the wrong rate, indicates a rating engine configuration error. The revenue assurance reports feed back to the mediation and billing teams as operational metrics.

For a large operator, closing a 0.01% revenue leak on a billion dollars of monthly revenue means recovering a million dollars per month that would otherwise be permanently lost. Revenue assurance is not an exotic function, it is how billing teams know their system is working correctly and how they surface problems before they appear on the regulator's radar.

Why This Is Hard in Practice

The telecom billing stack is hard for the same reason most financial processing systems are hard: the data volume is enormous, the rules are complex and change frequently, the timing requirements are strict, and the consequences of error are visible to customers and regulators. Any one of those factors would make it a challenging engineering problem. Combined, they make it a domain where depth of understanding matters more than technical cleverness. Most of the experienced engineers I met in this space had spent five to ten years in telecom billing specifically, and their knowledge of the edge cases, CDRs with partial timestamps, rating engine interactions between multiple concurrent bolt-ons, mid-cycle plan changes that affect mid-cycle rated events, was the accumulated result of working through real problems over many years. That's the kind of knowledge that doesn't transfer from a tutorial.