Looking back at 2021 with IPinfo - IPinfo.io blog

So long, 2021! IPinfo’s eighth year was marked by productivity and growth. If we had to sum this year up in a few words, here’s what we’d say: More data, better user experiences, and even more success stories.

Even though the data industry experienced some shifts this year (Apple Private Relay, the future of 3rd party cookies, etc), we were fortunate to have a team that responds quickly and adapts to the changes. Because of this, our team has been able to help innovators develop use cases for better fraud prevention, targeting, website personalization, IP anonymization, and more.

But what sets this year apart is an even bigger focus on data quality. If you’ve been around IPinfo for long, you know that our mission is to constantly improve our data. This year, we’ve exceeded our own high expectations when it comes to accuracy.

There are two main things we're always doing at IPinfo. The first is making sure that our data is great quality. That will be a project that will probably last forever. The other piece is how we make it easy for people to access that data. – Ben Dowling, Founder & CEO of IPinfo

Over the last year, we’ve developed more user features and tools while also improving our infrastructure even better, and more. We’ve bridged new partnerships with better integrations. And best of all, we got to talk with many of you - learning from current users, hearing more stories, and using your feedback to make IPinfo a better IP data provider.

So here’s to remembering all that you’ve contributed to IPinfo and all the data improvements we’ve made in 2021!

Data

This year we improved our data accuracy in several ways. But before diving into all the specifics, here’s a snapshot of our data throughout the course of a year.

Robust traceroute systems

As most of our users probably know, our database gets updated every 24 hours and all these updates are curated by our data team. We constantly check (and recheck) the IP data that enters our database.

Additionally, during 2021 we built an even larger network of servers to collect traceroute timings. Using triangulation, our team finds or checks IP geolocation. This has led to improved geo-accuracy. It also allows us to derive other uses, such as creating confidence bounds on the geolocation or detecting potential VPN users.

This traceroute system allows us to evaluate IP information from multiple perspectives, including these.

Pinging IPs
Evaluating the way traffic goes through different servers
Checking the typical speed of traffic
How much time waiting for responses from nodes
How many hops it takes to reach the IP

Currently, we're working on running our traceroute systems better (and more often) to improve coverage of IPs while also keeping our data as up-to-date as possible.

Our new data page: Making a (somewhat) invisible product visible

The reality is that IPinfo spends a lot of time making sure our database is full of value for our users. But here’s just one challenge facing data providers: helping users understand our multi-step, complex system of data retrieval and verification… without boring them to sleep.

To illustrate the value of verified IP data, our team designed a Data Page highlighting our process of data retrieval, data expectations, tools, and more.

For more details about how we process more than one petabyte of data in a year while keeping our database clean and accurate, check out this recent article.

New: IP WHOIS API

Another exciting development is our recent release of the IP WHOIS API. We developed this API in response to some requests for access to more WHOIS data. And within a short time of its release, over 100 users subscribed to get access to this specific dataset.

Here’s a little more background about this API’s design. We gather this data from over 400 servers. And our well-parsed WHOIS data allows users to programmatically retrieve the following insights by IP, ID, or domain:

Domain name
Domain range
Whois ID
Organization
Whois status
Technical contact’s ID
Administrative contact’s details
Date when the domain was created
Expiration date
RIR/NIR source
Netblock name
Country code
Full raw Whois record
Date(s) when the domain was updated

More than anything, our WHOIS API is an example of how feedback from our users helps us deliver better datasets. So thank you!

Apple Private Relay

We’ve also improved our data by staying current with industry trends. After Apple launched Private Relay (APR), our team noticed a growing percentage of APR users.

Unlike other Privacy Detection data (such as VPNs, tor usage, and proxies), iCloud Private Relay is geo-aware. Therefore, our users needed to distinguish APR traffic from other Privacy Detection insights.

In addition to working with Apple on adding these insights to our dataset, we also added specific attribution for APR in the Privacy Detection API. Now paid Business Plan can use these geo-aware datasets to improve use cases as APR traffic continues to increase.

Integrations

It’s been an exciting year when it comes to integrations, too!

Official IPinfo CLI launch

For starters, we launched the official IPinfo CLI, allowing easy access to almost all of our APIs and features. Now our newest tools and updates quickly make their way into the CLI. This allows us to offer even more support for developers while implementing user feedback more efficiently.

For instance, developers can now look up these details using our CLI:

Lookup IP details in bulk or one-by-one.
Lookup ASN details.
Summarize the details of up to 1000 IPs at a time.
Open a map of IP locations for any set of IPs
Bonus Feature #1: grepip for filtering IPs
Bonus Feature #2: prips for listing multiple CIDRs or an IP range

In the end, users can download CLI binaries (e.g. grepip, ipinfo) from multiple mechanisms. All they need to get started is our installation guide.

New Snowflake datasets and market integrations

Our team has also been busy creating new ways to access IPinfo’s data in Snowflake. Leveraging Snowflake’s powerful UDTFs, we’ve made our data more performant than ever before.

With our new join_key, users can enrich any IP address in Snowflake with these datasets: geolocation, company, carrier, and VPN detection data. All it takes is a simple call to our function. This new key can scale to massive datasets as represented by this graph.

The end result is that users never need to do a basic join. UDFs are just as fast and are even easier to read.

Graylog partnership

Back in May, we also announced our partnership with Graylog, a global provider of next-generation log management solutions. The founder and CTO of Graylog, Lennart Koopmann, wanted to offer an automatic way to enrich IP addresses with additional insights while developing Graylog Cloud.

That’s when they integrated IPinfo’s data into Graylog Cloud, offering all the functions of Graylog Enterprise with seamless data collection, rapid search, flexible analysis, and greater affordability.

This partnership reduces data latency that comes from manually accessing individual databases and conducting daily updates. IP addresses in log data can now be turned into additional insights, such as physical location. In short, this partnership improves user experiences and IP data usability - all things we’re pretty passionate about at IPinfo!

Better user experience and infrastructure reliability

If all the data improvements and integrations weren’t enough, our team also took time to improve user experiences and infrastructure reliability.

Goodbye, App Engine: Hello, Google Kubernetes Engine!

Over the years, IPinfo relied on Google App Engine to power and scale our applications. But as our traffic grew so did our need for more control and observability.

We eventually decided to transition to managing infrastructure in a non-traditional sense (i.e. without needing to keep operative systems up to date, worrying about per-VM configuration, etc.). If you want the full story, check it out here.

To make a long story short, though, we chose Google Kubernetes Engine (GKE) for its level of control on workload execution, autoscaling, and options for how traffic gets routed.

Our team planned the transition to GKE step-by-step. In the end, we successfully gained better infrastructure control with GKE while migrating our system without any downtime. IPinfo’s backend systems run even more predictably against traffic spikes and also give our team more visibility with better logs and metrics.

Website improvements

During 2021, our website also took a few steps forward. For starters, we developed an even more intuitive dashboard. User access to datasets, such as database downloads, add-on datasets, or bulk uploads, is even easier than before.

We also developed new landing pages for our use cases, case studies, tools, and more. Now our website is easier to navigate, and users can quickly find the information they need when they need it.

New IP tools

We also developed some new user-friendly tools, including some visualization features. Map IPs allows users to paste up to 500,000 IPs to view their location on a map. This visualization can be easily shared and is also one of our CLI commands.

Summarize IPs is another tool designed with user feedback in mind. This feature gives users a visual overview and insights for a list of IP addresses, including the following:

Location
IP types
Privacy statutes
Top ASNs and companies they belong to
Top countries and cities

These tools, along with our other improvements and infrastructure management, represent IPinfo’s long term commitment to making IP data more user-friendly and accessible.

IPinfo’s team

2021 also initiated some significant team growth. Simply speaking from a numbers perspective, our team grew from fourteen to twenty two during the course of 2021.

We added more data engineers, designers, web developers, and customer success specialists. All this to say, it’s been awesome to see our team expand.

Additionally, our founder and CEO, Ben Dowling, made some guest appearances on several podcasts and other events.

Guest Speaker at a Graylog Event
M2X: Tech Marketer Podcast, hosted by Adam Forziati
The SaaS Club’s Podcast with Omar Khan
SaaS District Podcast with host Akeel Jabber
Geomob Podcast with Ed Freyfogle
Natfluence interview

If you haven’t had a chance to listen to the podcasts yet, go check them out!

Our customers: IP data success stories

One of the best parts of being an IP data provider is hearing your success stories. And this year we’ve come across so many positive experiences that were worth sharing.

We also developed content highlighting your successes. These case studies range from ABM to data enrichment, cybersecurity to travel tech. Here’s just a sampling of the success stories we heard about this year:

Additionally, we’ve developed more user guides, like “How to filter IPs” and “How to create a simple rate limiter in Node.js”. All this to say, your stories and feedback have inspired our team and our users with the opportunities that IP data represents.

Thank you!

This year IPinfo’s data fuelled more businesses around the globe. Among them are Terminus, Bombora, Venmo, Tencent, Daimler, Cisco, and Pixar.

For the sake of every user, we’re committed to making IP data more usable and reliable, but we couldn’t do it without your support. Your feedback and suggestions were a valuable resource for the improvements we made in 2021.

In fact, we’d love to hear from you in 2022! If you want to say a quick hello or drop a few suggestions, send us a message on Twitter or on our contact page.

From the whole team at IPinfo, we want to wish you a great year ahead!