CRM, Email and Invoicing: Integrating Data Without Creating Duplicates

Connecting your CRM to your email platform and your invoicing software is, technically, the easy part. All three have APIs. Give it two afternoons and data is moving in both directions.

The problem shows up three weeks later, when the same customer exists four times and nobody knows which record is the real one. Data integration rarely fails at the connections. It fails at identity: at the decision, taken before the first automation is written, about what makes two records the same person.

Four records, one customer

A bakery buys services from you. Here is what your systems stored.

• In the CRM: "Silva Bakery Ltd", created by a salesperson, with the owner's personal email.
• In the invoicing software: "SILVA BAKERY LTD", created by finance, with a VAT number and a registered address.
• In the email platform: hello@silvabakery.com, imported from a website form.
• And again in the email platform, in a second audience, the owner's email, collected at a trade show.

No system is wrong. Each one stored what it needed. The CRM wants a person to talk to, invoicing wants a legal entity, the email platform wants a deliverable address. Those are three different definitions of "customer", and all three are legitimate.

The cost appears when you join them. Mailchimp's own help centre is explicit about it: if you send the same campaign to two audiences, a contact who sits in both receives it twice. There is no warning. The customer is the one who notices.

The more expensive effect, though, is internal. When nobody trusts the record, everybody re-checks everything by hand before acting — and that is where a large share of rework between teams comes from, without ever showing up in a report.

Pick the key before you build the automation

Every data integration rests on a key: the field that answers "does this already exist?".

In B2C, that field is almost always the email address. It is unique, the customer types it themselves, and it exists in all three systems.

In B2B the obvious answer is the company tax or VAT number — and that is usually where it goes wrong.

A VAT number makes an excellent key for business customers. It is issued by the state, it survives changes of address and trading name, and the invoicing software already requires it. But the exceptions are country-specific and they bite. In Portugal, invoices issued to a walk-in consumer carry the generic tax number 999999990, the placeholder every certified invoicing system uses when the buyer does not give their own.

Use that field as a key without handling the exception and every private customer collapses into a single giant record called "Final Consumer". The result is not a duplicate. It is the opposite of a duplicate: hundreds of people merged into one, with all their purchase history mixed together. And unlike a duplicate, it never looks wrong in a report.

The practical rule is a hierarchy, not a single key:

1. The tax number, if it exists and is not the generic placeholder for your country.
2. A normalised email address, when there is no usable tax number.
3. If neither is available, create a new record and flag it as unconfirmed. Never guess.

That third point is what separates an integration that ages well from one that slowly rots. Merging two records because the names look similar is quick to build and almost impossible to undo.

Validate the key, do not trust it

A key is only a key if it is correct. A mistyped VAT number creates a brand new customer just as reliably as it would create a duplicate.

Two checks handle most cases, and both run in milliseconds.

The first is the check digit. Most European tax numbers carry a checksum: the Portuguese NIF, for instance, has a final digit computed from the eight before it. A typo fails that arithmetic almost every time. It is ten lines of code and it catches the problem before it enters the system.

The second is external validation. For business customers, the European Commission's VIES service confirms whether a VAT number is registered and, depending on the member state, returns the official name and address. That serves two purposes: rejecting invalid numbers, and filling the record with the official trading name instead of whatever variation the salesperson typed in a hurry.

It is worth knowing what VIES does not do. It is a public service with no API key, with per-country outages and no response-time guarantee. Any automation that depends on it to save a record has to tolerate failure: store the record, mark validation as pending, retry later. Blocking customer creation because an external service is down trades a small problem for a bigger one.

For email, normalising means little more than trimming whitespace and lowercasing. Resist going further. Stripping dots, or everything after a plus sign, works on Gmail and is wrong on plenty of other servers — it will merge different people.

Upsert, never create

This is the technical change with the highest return and the least discussion.

An automation that calls "create customer" creates a customer every time it runs. If the flow repeats because of a timeout, an automatic retry or a webhook delivered twice — ordinary events, not exotic ones — you end up with two identical records.

The alternative is the upsert: write against the key, not against the system. "If a customer with this VAT number exists, update it; if not, create it." The operation becomes idempotent. Running it ten times produces the same result as running it once.

Salesforce's REST API documentation describes the mechanism well, including a detail that usually goes unnoticed: if the key matches more than one record, the operation fails with an error and writes nothing at all.

At first glance that looks like a flaw. It is the opposite. It means the system refuses to decide on its own when the data is ambiguous, and hands the case back to a human. An automation that fails loudly once a month is far better than one that quietly duplicates every day.

Not every tool exposes a native upsert. Plenty of invoicing packages force you to search first and create afterwards — two calls instead of one. That works, as long as the search runs on the key rather than the name, and as long as the flow handles two simultaneous requests for the same customer.

The duplicates already sitting there

Here is a constraint that rarely gets mentioned and that changes the whole clean-up plan.

In invoicing software certified by a tax authority — the norm across most of southern Europe — an issued document cannot be deleted or altered. That immutability is precisely what the certification guarantees. The practical consequence is direct: you cannot simply merge two customer records that have already issued invoices, because those documents stay bound to the record that issued them.

What you do instead is duller. Choose the canonical record, mark the others inactive, block new documents from being issued against them, and keep a mapping from the old identifiers to the new one, so historical reports still add up.

In the CRM and the email platform you have more freedom. But even there, merge with a log of what was merged. A merge without history is irreversible, and sooner or later somebody will need to reverse it.

What the tool solves, and what it does not

If you are building this on an automation platform such as n8n, the Remove Duplicates node handles the execution layer: it discards repeated items inside the same batch and, in "Value Is New" mode, remembers values already processed in earlier runs.

It is useful, and it has limits worth knowing before you rely on it. By default it stores 10,000 items of history — past that volume the oldest entries drop out and an old record can be treated as new again. And that history belongs to the node by default, not the workflow: two separate nodes share nothing unless you switch the scope to "workflow".

The distinction that matters is this. That is execution-level protection, not identity management. It stops the same event being processed twice. It does not stop two different events creating two records for the same customer. For that, the key has to live in the destination system, with a uniqueness constraint — not in the platform doing the orchestration.

What it costs to leave this alone

Simple arithmetic, with the assumptions in plain sight. This is not a study; it is a sum you can redo with your own numbers.

A company with three systems and roughly 1,500 customer records:

• Before each campaign, somebody exports three files and reconciles them in a spreadsheet: 3 hours.
• Ten campaigns a year: 30 hours.
• Invoices issued with stale details and corrected afterwards: two a month, 20 minutes each, credit note and phone call included. That is 8 hours a year.
• Total: about 38 hours a year on work that produces nothing.

Thirty-eight hours do not justify a data management programme by that name. They justify half a dozen days spent defining the key, adding validation and swapping create for upsert. To turn those hours into money using your team's real hourly cost, the method is in the article on what manual processes actually cost.

And the larger cost never enters that sum: the decision taken from a report that counts the same customer three times.

Where to start

Do not start with the tool. Start with a question you can settle in a twenty-minute meeting: which field, in your company, decides that two records are the same customer?

If the answer is immediate and identical for everyone around the table, the data integration will be simpler than it looks. If everyone gives a different answer, you have just found the problem — and it is not a software problem.