An Introduction to FST Network’s Logic Operating Centre (LOC)
A Non-Intrusive, Logic-Driven, Cloud-Native Data Governance Solution for Enterprises
Author’s note: the technological details described in this article is based on an older version of FST Network’s LOC. Since LOC is still activity evolving to meet our customer’s needs and to stay competitive, please refer to our official blog and documentation for latest updates.
The “competitive advantage” provided by data may be monetization of the data; finding ways to turn data insights into real revenue. Data can also help position a company to provide a better client experience, more effective product pricing and targeted marketing programs, ultimately improving business performance.
To achieve these goals, however, requires the data to be understood, authorized, and accounted for — in other words, governed.
— IBM Blog: Unified Data Governance: A Competitive Advantage
Imagine: you are in charge of the sales department of an insurance company. A new insurance policy product is developed for an emerging market. The IT department has created a database for the insurance system but it is not yet hooked up with your legacy accounting system.
You also find out the new product requires additional data from some systems to comply government regulations, but you don’t know exactly where.
After making a bunch of phone calls and hold many meetings, the data officer locates all the right data and the dev team put together some workable services. That will do…for now.
Whenever there’s a need to create a new data process, the company has to reinvent the wheel all over again. The number of heterogeneous grows, any new system addition would only be more time-consuming and chaotic. It takes forever to find data among dozens or hundreds of systems. The company starts to lose the competition because they are not able to satisfy their customers anymore.
Your CIO think about this and realize that the company needs a unified business intelligence (BI) system. You need to find one system that can govern them all.
Meet LOC, Your Virtual Data Officer
Data governance is a principled approach to managing data during its life cycle, from acquisition to use to disposal.
Every organization needs data governance. As businesses throughout all industries proceed on their digital-transformation journeys, data has quickly become the most valuable asset they possess.
— Google Cloud: What is data governance?
FST Network’s Logic Operating Centre (LOC), is a revolutionary new data governance solution for enterprises. LOC would create a abstract “logic intelligence layer” between your existing systems/data and connect them with data processes.
The idea of the logic intelligence layer can also be explained from the angle of data mesh, a popular new data model. Check out the article link at the end of this post.
The data processes (or data pipelines) are consisted of one or multiple data logic:
- Data logic are based on business logic designed by business analysts (BA).
- System analysts (SA) do information system analysis for BA and have developers implement data logic code via LOC. A series of data logic are linked into a data process.
- LOC allows data logic to connect services and data sources via LOC “agents”.
- Event sourcing: data processes can be triggered (executed) by events, message queues (MQ) or schedulers.
- Output data can be accessed via a API endpoint or message queue. Data logic can also write data into files or databases during the process.
LOC is designed to be non-intrusive. That is, it changes nothing of your existing systems. It instead offers an opportunity to build a streamlined bypass over your current data operation. You can even gradually integrate LOC into your company, build new systems and infrastructure around it.
The ultimate vision of LOC is, of course, to become our client’s virtual assistive data officer (VADO) — a 24/7 data engine that eliminates the need for hiring human data officers and provides an unified, virtualized layer for accessing data.
While the BFSI (banking, financial services and insurance) industry is going through incredible growth, the increasing complexity of the various data pipeline demands modernization of back-office operations. FST Network’s technology is the platform we need now to connect people, data and processes within Financial Institutions.
— Jack Chu, CEO of FST Network
Back to our example in the beginning: by adopting LOC, our insurance company will be able to build data processes in a much quicker, painless fashion. It’ll have to link data and services for the first time, sure, but a process can be copied and modified without having to find data again. When the data is already in place, it’s only a matter of “connecting the dots”.
FST Network also offers BA and SA templates or consultancy upon request (see the contact info at the end of this article).
What else does LOC offer?
- “Low code” —70% less code; only the data processes themselves need to be implemented
- “Logic mesh” — creating a network of data products
- Data logic/processes versioning, ownership control and audit logging
- Data discovery: event browsing/filtering and data lineage visualization
- Exporting results to database and files
- A strong support by FST Network’s product and dev team
LOC is Built With Cutting-Edge Software Technologies
Now we will talk a preview of the technology side of LOC. The Logic Operating Centre has three major components:
- LOC Core — the data governance engine (written in Rust)
- LOC Studio — a web-based user interface/management system (front-end is implemented with React + TypeScript and back-end is also written in Rust)
- LOC CLI— command line tool for developing/uploading/executing data processes from a local machine (developed with TypeScript)
The data processes themselves are mini-applications created by the LOC Core and will be run in the Kubernetes cluster. Our clients can implement the data logic code either with JavaScript or TypeScript in LOC Studio and/or LOC CLI.
LOC automates a lot of behind-the-scene operations, so you don’t have to worry about how data processes are actually deployed. For more experienced developers, LOC CLI enables them to develop data processes from their own machine.
Can we do local test though? The FST dev team actually also provides a local LOC environment that you can install on your machine, which can be used to test data processes without uploading!
What does our software stack mean to LOC ?
FST dev team has carefully chosen a stack of capable new technologies to built LOC. These plays a crucial role in LOC’s high performance and reliability.
In our future blog articles, we will discuss more in detailed subjects. But here’s some quick summaries:
- Google’s Kubernetes is a proven cloud-native app management system. The so-called container orchestration enables LOC as well as data processes to run with incredible reliability and scalability. LOC can be deployed literately anywhere as well thanks to Kubernetes.
CNCF (Cloud Native Computing Foundation)’s State of Cloud Native Development Report for Q1 2021 indicates that 1 in 5 developers (5.6 million people) globally is using Kubernetes. In Stack Overflow Survey 2021 it is the third most loved tool after Git and Docker.
- Rust is one of the fastest system programming language today, also guarantees extremely stable production code as well as being computing-efficient (smaller memory usage and less energy consumption). By using Rust, LOC has the edge that only few other systems possess and can reduce operation cost at the same time.
Rust has been voted as the most loved language by developers in Stack Overflow Surveys 2016~2022 — seven years #1 in a row.
- Microsoft’s TypeScript is an “augmented” version of JavaScript, which has lots of built-in safety features for developers to reduce debugging time. The scripting language is relatively easy to learn — hence we picked it as our “data process language” for clients.
TypeScript is the third most loved language in Stack Overflow Survey 2021 and ranked #4 in The 2021 State of the Octoverse (Github Survey). You can still use JavaScript if you like — it’s one of the most popular languages of all time.
- But don’t worry — LOC supports JavaScript as well! It’s the most popular programming on Earth!
- Finally, Facebook’s React is one of the modern reactive web framework pioneers and still a prominent one, an excellent choice for building rich-user-experience interfaces. It is the front line that LOC users would be most familiar with.
React is the most used front-end framework (80%) by front-end developers in the survey State of JS 2021, over Angular (54%) and Vue.js (51%).
Also, all these tools are open-sourced — you don’t have to pay extra for proprietary software.
Thanks for reading and stay tuned! We will publish more about technology topics on our Medium blog.
Related Reading
This article was written with help from FST Network’s product and dev team.
For more info about Logic Operating Centre (LOC) or demonstration request, please contact support@fstk.io.
