Data integration

illustreets processes geospatial and enterprise data, and delivers business analytics apps to your clients.

It is a cloud service, built and hardened to run on the web, but can also be placed inside your local network, or in a virtual private cloud (VPC). It has a data integration workflow designed to be secure out of the box, but without trading off speed, flexibility, and usability.

Fast ingestion without compromising security

Data can be imported in three ways:

  1. Manually uploading local files (e.g., CSV, KML).
  2. Fetching files from various web services*. You can schedule this to run periodically.
  3. Using our standalone data integration service, and fetching data from 20+ sources.

Whatever the method, illustreets enforces communication over secure channels without exception, to mitigate the risk of eavesdropping. Our TLS setup achieves the A+ rating on SSL Labs. Furthermore, data imports can be set to run over private connection, with AWS PrivateLink or Azure Private Link.

When using our integration service, if expecting to connect to production databases, we recommend installing it within a secure perimeter, and pushing the output to an encrypted simple storage service with access logging. If there are existing integration/transformation services in place, such as Alteryx, we suggest utilising an object storage service for unifying the output of these tools and routing it to illustreets.

If required, maximum control over the data flow and complete privacy can be achieved by disabling manual uploads, and limiting backend connections to the object storage service only.


* Out of the box, illustreets can fetch data from: ArcGIS Server, GeoServer, Carto, Amazon S3, Azure Blob Storage, Dropbox, Google Drive, Google Sheets. Also, any service offering direct download over HTTP(S) is supported, e.g. Esri Open Data Hub, Natural Earth Data, Data.gov, Socrata Data Platform, etc.

Protecting data in transit and at rest

In illustreets, the risk of unauthorized access is mitigated through a combination of role-based access control (RBAC) and attribute-based access control (ABAC). This capability allows you to fine-tune the team members’ access permissions: who is allowed to access raw data, who builds visualizations, who can add/modify users, who designs reports, and so on.

To protect against outside breaches and internal fraud, user actions are being logged by the system in the form of an audit trail. Our strict HTTPS-only policy encrypts data during transit at all times. On request, we can also enable database at rest encryption, to further protect your data against intrusion and theft.

Ensuring data integrity and availability

Analysts are able to combine data, automate transformations and analysis, and link-up data and visualizations all the way to the front end. Given this continuous data flow, we enforce automated integrity checks, to minimize the potential for error and boost the user experience.

Data is being continuously checked and validated against its original schema automatically, throughout its entire lifecycle. This enforces document similarity and prevents inconsistencies in front-end apps. Nobody wants a dashboard in the boardroom full of NaNs or NULLs because some ETL process went wrong and mangled the source data…

When a dataset receives updates, all its descendants — data views, maps, charts, and so on — are updated in succession, all the way to the front end. This enables a degree of effortless automation rarely found in other systems. Though, if you have data sources known to suffer from inconsistent quality, you can create “data buffers” in the form of cached views, so the new data can be checked manually before the updates are allowed to propagate further.

Standalone data integration service

We offer a data integration service, which can be installed on request. The options are: 1) allow full integration with illustreets, whereby each analyst can create/modify sources via illustreets’ interface, or 2) run it standalone and define a limited set of data sources within the service.

It can be installed on a separate network, on-premises, or in the cloud. You can choose to have the service managed by your network team, or by illustreets. For increased security, you can consolidate the integration workflow, and any other delivery pipelines you may be running internally, using an object storage service like Amazon S3.

CONNECTORS

Out of the box, our data integration service supports these connectors. More can be added upon request.

Data warehousing
  • BigQuery
  • Redshift
  • Snowflake
Database systems
  • CockroachDB
  • ClickHouse
  • IBM Db2
  • Microsoft SQL Server
  • Mongo DB
  • MySQL
  • Oracle DB
  • PostgreSQL
Web services
  • Google Sheets
  • Freshdesk
  • Hubspot
  • Quickbooks
  • Salesforce
Static files - sources
  • HTTPS
  • SFTP
  • SSH / SCP
  • Amazon S3
  • Azure Blob Storage
  • Google Cloud Storage
Static files - formats
  • CSV
  • JSON
  • XLS(X)
  • XLSB
  • Feather
  • Parquet
CSV files - geometry
  • Polygon/line geometry column as WKT
  • Point geometry as lon/lat or x/y columns
  • Fully supporting the GeoCSV specification

Mapping your success

Seamlessly import and integrate your data to create interactive maps, insightful dashboards, and powerful business applications. Say goodbye to static spreadsheets and hello to dynamic, collaborative, and ready-to-deploy business applications that add real value to your enterprise.

Book a demo

What our clients say

illustreets — a world-class product that outperforms industry giants, coupled with personalized service” and “a group of intelligent and friendly individuals to work alongside”. “We have only scratched the surface of the potential [it] provides and it’s been a great experience using it”.

illustreets has played an integral part in helping us build our SaaS platform for Terravault.

They worked collaboratively with us to develop tools and functionality that enhanced our product. In addition to their expertise, they are a group of intelligent and friendly individuals to work alongside.

Vicki Burch-Vasquez, GIS Manager
Vicki Burch-Vasquez, GIS Manager
Cude Engineers, USA

It’s not often one comes across a company like illustreets — a world-class product…

…that outperforms industry giants, coupled with personalized service that allows product customization to allow exploration of new markets.

Village Infrastructure Angels has combined its knowledge and illustreets to deliver world class infrastructure mapping and planning services that, to the best of our knowledge, has no comparable competition, resulting in repeat work captured for both companies from respected institutions like the World Bank, Inter-American Development Bank and many others.

It has been a sheer delight to work with the illustreets team, and I look forward to many more years of collaboration that can result in more infrastructure being built to help the world’s poorest 1-2 billion people, and probably saving governments and donors billions of dollars via large-scale innovative design.

Stewart Craine, CEO
Stewart Craine, CEO
Village Infrastructure Angels, UK

We replaced spreadsheets of coordinates and calculation tables with illustreets.

Our business is selling equipment, which brings least cost electrification out to hard to reach areas, so we took our geospatial plannning data and plotted it in illustreets.

It became quickly apparent to our partners that they could make massive savings by setting up decentralised clusters of mesh-grids vs. more traditional centralised solutions. They were able to show this to other engineers and management in their organization and quickly we were able to move to supplying equipment and getting more power to the people.

We have only scratched the surface of the potential illustreets provides and it’s been a great experience using it. Highly recommended easy UX - quick leap to visualising GIS data.

Afnan Hannan, CEO
Afnan Hannan, CEO
Okra Solar, Australia

illustreets has provided an unmatched ability to build on large and complex datasets…

…to not only make our data visually beautiful but to present complex data as simple and easy to understand dynamic stories to our clients. This is particularly important to us when dealing with highly technical data to enable non-technical clients the ability to easily understand and navigate their way through the GIS apps, without getting lost in a mass of data and information. Previous solutions we had worked with lacked this final piece of the puzzle.

In addition, the SQL engine within illustreets is unrivaled in it’s ease of use - as a relative novice in SQL language, it was easy to get to grips with utilising some of the power of this engine using their simple and user friendly SQL query builder GUI and later move into more advanced queries with the SQL editor.

The support illustreets has provided throughout has been first class and always met with quick responses, even when the support requested has not necessarily been related to illustreets platform!

Sam Andrews
Sam Andrews
Andrews Consulting, Australia