Engineering @Zeta

Zeta Tech Stack

Zeta is in the business of providing a full-stack, cloud-native, API-first neo-banking platform including a digital core and a payment engine for issuance of credit, debit, and prepaid products. These products enable legacy banks and new-age fintech institutions to launch modern retail and corporate fintech products. 

Zeta currently provides its platform and products to BFSI issuers in India, Asia, and LATAM. Zeta’s products are used by banks such as RBL Bank, IDFC First Bank, and Kotak Mahindra Bank, 14000 corporates, and over 2 million users. Zeta is a SOC 2, ISO 27001, ISO 9001:2008, PCI DSS certified company.

How does Zeta continuously deliver at these high levels without compromising on security? Let us take a look at Zeta’s tech stack to find out.

Server Applications

A server application is designed to install, operate, and host applications and associated services for end-users.

Compute Runtimes

Technology usedKubernetes, GitOps, Apache Openwhisk, Apache Flink, and Camunda. 

Following are the runtime applications in Zeta.

  • Atlantis is a docker runtime container deployed using GitOps based end-to-end orchestration with the help of Kubernetes.
  • Aster is an open-source serverless platform run on Apache Openwhisk that runs various short-lived jobs on-demand. 
  • Rhea is an application used to execute BPMN workflows that run on the Camunda platform.
  • Perseus is an adaptation of Apache Flink to run complex event processing tasks. Apache Flink is a framework and processing engine used for computations at any scale.

We use Camunda to define workflows and automate decision-making. 

Apache Flink provides a high-throughput, low-latency streaming engine and supports event-time processing and state management. It has allowed us to perform computations at any scale.

Cluster Management

Technology usedKubernetes, Kong, and Calico. 

A cluster management tool is an essential service used to group the clusters and nodes. This is used to:

  • Monitor nodes in the cluster.
  • Configure services.
  • Administer cluster services. 

We prefer using open-source systems in our tech stack. That is why we use Kubernetes as our orchestration system to efficiently manage and deploy containers. 

Ingress and API Gateways run using Kong. Kong supports high availability clusters and an extensive range of plugins to address various concerns like authentication, security, and monitoring.

We have also developed 2 in-house tools, Sprinkler and Hades. Sprinkler is our Egress gateway and Hades is our file input/output gateway. 

We use the Calico plugin to manage the container network interface.

We are evaluating Istio to adopt it as our open service mesh. A service mesh controls how different parts of an application share data with each other.

Messaging Infrastructure

Technology usedApache Kafka, Apache Nifi, Apache Flink, KSQL, and Kafka.

A messaging system transfers data from one application to another. We use the following components to seamlessly transfer messages within Zeta’s various applications and clusters.

  • Apache Kafka is used as a message broker. It allows us to handle high volumes of data and passes messages from one end to the other. 
  • Atropos is a message-based integration bus based on Kafka developed by Zeta.
  • Apache Nifi is used to implement ETL Pipelines. 
  • Perseus, developed by Zeta, uses Apache Flink to process complex events. 
  • Sinope uses KSQL and Kafka connectors to bridge the message/event world and the batch/file world. KSQL and Kafka connectors enable rich data integration and streaming.

Data Stores

Technology usedPostgresql, Amazon S3, Amazon RedShift, ClickHouse, and Redis.

Postgresql is used to store transactional data and Amazon S3 is our data lake. We use Amazon RedShift and ClickHouse as our data warehouse and Redis, an open-source memory data structure, to store cache.

Application Performance Monitoring (APM) and Business Performance Monitoring (BPM) uses the ElasticSearch engine.

Data Processing

Technology usedPresto SQL, Sparta, and Apache Superset.

The data processing unit is an essential part of any server application. It converts data available in the system (machine-readable data) to a human-readable format. 

We use the Presto SQL query engine to process big data and Sparta for stream processing. Stream processing allows us to convert data in motion directly to continuous streams. 

The Power Centre stores various data models and allows us to modify data and enables the reuse of data. The Power Centre is an adaption of Apache Superset.

CI/CD

Technology usedJenkins, ArgoCD, and Sonarqube.

Jenkins is used for Continuous Integration (CI) and ArgoCD for Continuous Deployment (CD).

Our servers use Sonarqube, an open-source platform, for static code analysis.

Observability

Technology usedGrafana, Kibana, Fluentd, and Hypertrace.

Observability is defined as the ability of the internal states of a system to be determined by its external outputs. External outputs can be anything such as alerts, visual representation, and  infrastructure tracing.

We have developed Prometheus as our event monitoring system. Prometheus stores all metrics in the database. 

We use Grafana for our dashboards and alerts and Kibana for log analysis. The purpose of both these applications is to visualize and navigate data. We also use Fluentd, an open-source data collector, for logs and metrics collection.

All applications should have a distributed tracing system. Hypertrace is our tracing system. It helps us debug applications and log information about their execution.

Security Monitoring

Technology usedHELK and Jackhammer.

Security monitoring is another essential unit to analyze data to detect any suspicious behavior or changes to the network. 

HELK is an ELK (Elasticsearch, Logstash & Kibana) stack with advanced hunting analytic capabilities provided by the implementation of Spark and Graphframes technologies.

Kratos, an in-house adaption of Jackhammer, is our Security CI.

We are evaluating RockNSM, a premier sensor platform for Network Security Monitoring (NSM) hunting and incident response (IR) operations. 

Hardware Security Module (HSM)

Technology usedSafenet, Gemalto and nCipher/ Entrust nShield Solo.

HSM manages digital keys, encryption and decryption, and provides strong authentication. We use the following HSMs:

  • Safenet and Gemalto as Network HSM.
  • nCipher/ Entrusy nShield Solo.
  • Harpocrates, developed by Zeta, is our HSM as a service interface. 

Mobile Application

Technology usedFirebase Hosting, Websockets, HTTPS.

Languages usedKotlin, Java, Objective C, Swift.

We develop mobile applications as native apps, React Native apps, and web apps.

Kotlin and Java are used to develop our Android applications and Objective C and Swift for our iOS applications.

Firebase Hosting helps with Analytics and Crash detection in the server. Websockets and HTTPS make it possible for two-way communication between the server and the user.

Web Applications

Technology usedTypescript, SCSS transpilers, Vue.js, Bulma, Buefy, Webpack, Rollup, Verdaccio, Lerna, and Sentry.

Languages usedHTML, CSS, Javascript, Node JS, and Express JS

We use HTML, CSS, Javascript, Node JS, and Express JS to develop our web applications.

Typescript and SCSS transpilers convert program code from one language to another.

Vue.js is our Javascript framework and Bulma is our CSS framework. Buefy is a user interface component and is made using both Bulma and Vue js. Webpack and Rollup are the JavaScript build tools. 

Verdaccio, an NPM registry, serves the purpose of a Private NPM repository. The repository is managed by Lerna, a workflow optimization tool. Sentry, an Application monitoring tool, is used to monitor errors.

Cyber Security

Technology usedTrivy, Clair, CloudSploit, MobSF, ZAP

The challenge when building a product is how we balance ease of use with security. At Zeta, we use Trivy for CI docker scanning and Clair for CD docker scanning.

CloudSploit is our Cloud Security Posture Management.  We use MobSF for mobile application security.

ZAP for web applications is all in CI/CD.

Best Tech Stack = Best Performance?

Building a system that can provide these functionalities and process over 1 million transactions per second is not easy. While Zeta uses the best tech stack, building a robust framework to continuously deliver at the highest levels without compromising on security would not have been possible without Zeta’s talented engineers.

Today, Payments are generally insecure

Have you ever been asked by a waiter at a restaurant for your card PIN? Has he ever written it down on a piece of paper? Have you ever noticed a CCTV camera watching you enter the PIN at a shop? Have you ever used an app to buy something, and got redirected to your bank’s web page to enter the PIN? Have you ever given permission to an app to read all your SMS messages, some of which may contain payment OTPs? 

By now, you might have guessed that your PIN and OTPs are easily compromised.

The many technological advancements since the time card based payment systems were introduced have made the systems defenseless. Fundamentally, card based systems require every machine and all intermediaries involved in the transaction to be secure. There are way too many of those machines and intermediaries across the world to ascertain security of them. Many people who use those machines, like merchants, have little idea about their role in securing the entire system. Any breach at any one machine can cause unrecoverable damage to several users. The card details once captured can be reused several times for many transactions.

To overcome some of the limitations of the payment cards, a PIN is mandated for every transaction. A 4 digit PIN offers protection against many trivial threats. But given the reusable nature of the PIN and also the poor choice of PIN made by users, it is not a strong enough defense.

It is hard for the user to know if the card machine at a retail store or the ATM on which she is entering the PIN is tampered with. Banks providing these machines can’t practically keep a watch, as several machines are deployed in unmonitored physical environments.

In case of personal devices like phones, users will have far better visibility and control on tampering. However, most of these devices and the software that’s running on them are also vulnerable to a range of threats. Android users can easily notice that many programs can read the SMS messages on their phone. These programs can also export the OTPs in messages to a fraudster’s machine without user’s knowledge. Also, almost all mobile apps that accept card and netbanking payments can read and store the passwords, OTPs and PINs entered by the user. It may appear to the user that she is entering the PIN on the bank’s payment page. It is nearly impossible for the user to know if her password or PIN is captured and stored by the app.

It is not easy to trace the fraud back to the fraudulent app or merchant. The information captured by one app can be used by anyone from anywhere in the world. If one ATM is compromised, the cards used on that machine can be replicated and used on any other ATM. These vulnerabilities are just the low hanging targets for a fraudster.

To overcome vulnerabilities in operating systems, applications, hardware components and to defend against powerful machines and programs that can exploit the minutest of the vulnerabilities, it is essential to make security a foundational aspect of the system. The payment system providers should be fanatic about security.

Secure Foundation for a Modern Payment System

When we started work on Zeta in April 2015, we had the opportunity to consider a great expanse of devices and applications the modern connected world is currently using and is likely to use. We studied the state of the art secure protocols, services and algorithms and designed several core components of our payment system quite differently from that of legacy bank or card systems. We spent the first 3 months of our time in arriving at a secure architecture at the core. We used asymmetric encryption and signature algorithms to defend our systems against large scale fraud.

We arrived at the following ground rules to secure transactions against many of today’s threats:

  • 3rd party devices: System should not rely on any non-personal devices of the user for transaction security. The card reading machines, POS machines, ATMs or any other machines that don’t belong to either the transacting user or zeta should be assumed as vulnerable.
  • User provided data: Users are prone to use easily retrievable personal data for passwords, pins or for such other key material. System should not solely rely on such information to secure transactions.
  • Data Persistence and Transmission: Every transaction must require information that is never persisted on any device involved in the transaction. Information transmitted for a transaction should not be reusable to do any other transaction.
  • Visible Information: If information is visible to user, we assume that it can be shared or can be stolen through techniques like social engineering. No data the user may be able to share during the course of a transaction or otherwise should be sufficient to do a new transaction.
  • Data Location: No amount of data at rest on any one machine, or if possible in any one location (data center), should be sufficient to complete a transaction
  • Hardware Security Modules: Always use hardware security modules to secure the key material. On users’ devices that don’t support access to secure elements, use encrypted stores for key material.
  • Channel Sensitivity: System should strive to use the highest entropy algorithms and keys suitable for each transaction channel.

These rules guide the protocols and algorithms we use on Zeta. They influence many user interactions on our mobile and web applications as well.

Securing Card Payments

We provide Super Cards to many of our users. These are traditional plastic cards with magnetic stripe, provided to make payments to millions of card accepting merchants. This is our approach to make Zeta backward compatible with the present day payment systems. However, we didn’t want to limit ourselves to the insecure practices widely prevalent across industry. Using the ground rules discussed, we have adopted a different security model for Super Card transactions.

In-Store Payments


img_20161023_113108_664

Dynamic OTP on the phone can be used as PIN

When a user transacts using his Super Card at a retail store, instead of a 4 digit static PIN, she can use the dynamically generated Zeta Code as her PIN. Using the Zeta Code ensures that even if the card reading machine or any of the intermediaries are compromised, no fraudulent party can make use of the PIN for another transaction.  

Online Payments


Zeta SecureCode Page    img_20161023_114134_700

User need not enter OTP for online payments

When a user starts a payment on an ecommerce website, she will be greeted by our custom SecureCode* page. Parallely, she is prompted with a dialog on her phone with an option to ‘swipe to pay’. If the user wants to confirm the payment, she swipes the slider on the screen and enters her PIN through a custom secure keyboard provided on the phone.  The payment page on the browser proceeds to the merchant website with payment success. The user wouldn’t have entered any input on the SecureCode page in the browser. As the user need not read and enter the input, we can now use significantly more secure algorithms like digital signatures to represent authorisation for the payment. This approach also offers a better user experience and quicker payment completion.

If the user is offline on her phone when she is on our SecureCode page, she can generate a Zeta Code on her phone and enter that on the SecureCode page. Each such 9 digit Zeta Code is valid only for 2 minutes. This eliminates the need for static passwords, PINs and SMS messages and thus their vulnerabilities.

Security in Mobile App Interactions

We believe relying on user’s personal device is better for security than relying on any 3rd-party device.  However, it is important to defend the information on the device not only against the possible loopholes of the hardware and the software, but also from the user’s ignorance. Failing to do so could allow for exploits in the form of malware and social engineering.

Therefore, when defining the security model of Zeta apps we assumed that on users’ phones:

  1. Keyboards are compromised
  2. SMSes are compromised
  3. Data at rest is vulnerable
    (Even though stored in application specific stored provided by OS)
  4. Text labels on the screen are compromised

We built a model that can maintain security and integrity of the transaction against such compromised components. A specific example is the Zeta code that is generated on phone to make a payment.


img_20161023_120833_837     img_20161023_121133_893

  1. To generate a Zeta code, user has to press and hold on an area on the screen until a ring is fully drawn.
  2. When the device is online, the code is generated against a digitally signed request from the user. We use ECDSA to achieve reliable encryption strength.
  3. The private keys used for signing the code generation request are encrypted and stored on disk. On phones that don’t have a secure element backed user authentication mechanism, the encryption key is derived from user’s PIN. On phones that have a secure element on, the keys are generated using high entropy sources available on phone and are stored in a secure element backed storage+.
  4. When we ask for PIN, we take user input through a custom keyboard that obfuscates every key press. The labels of the on-screen keyboard do not represent the actual data captured on each key press. Thus, the PIN of the user is virtually never captured in the form user remembers or reproduces it. Also, the derived input from those key presses never leaves the phone. 
  5. Once the code is fetched from the server, although the code appears like text, it is rendered on screen as an image. No malware or screen reader in the device will be able to trivially parse the text on the screen. 

Summing up the Card Payment Security

We must acknowledge that the card based payments can be made significantly more secure than how they currently are. MasterCard, Visa and Rupay have provisions for better security, but many of the prevailing implementations do not make good use of these provisions. We hope all of the industry players will adopt more secure options in the wake of the recently observed 3.2 million debit card compromise.

Although the recent breach is related to in-store and ATM transactions, we also hope that it serves as a wake-up call to reconsider some of the insecure practices related to online card payments as well.

We must constantly strive to earn the confidence that our users place in us. In the payment industry, it is extremely hard to win back a user’s trust after it is lost. We at Zeta will be happy to help everyone in the industry in this endless and rewarding journey of building and running secure payment systems.

* SecureCode is a trademark of MasterCard.
+ Our Android/Windows apps currently do not recognise secure element. The data is protected using keys derived from user’s PIN.