In this article, I will take you through the process of how to backup Fluree Database Snapshots to Tardigrade using ASP.NET Core Service Worker(Background Service).

Introduction

What is Fluree?

Fluree is an immutable, time-ordered blockchain database. Each block is an atomic update that is cryptographically signed to prevent tampering and linked to the previous block in the chain. Fluree can be run privately or as part of a federated network.

A private Fluree is a database run on a single server, either by you or hosted for you by Fluree. A private Fluree gives you access to features such as time-travel queries and rich permission logic.

A federated Fluree is a group of databases shared by the network. In addition to all the features of a private instance of Fluree, having a federated group of databases provides additional data integrity. With a federated blockchain, the network uses an agreed-upon consensus algorithm to reach a shared state.

Check Fluree Documentation for more.

What is Tardigrade?

Tardigrade is an encrypted, secure, and cost-effective object storage service that enables you to store, back up, and archive large amounts of data in a decentralized manner.

Tardigrade has some great features tailored to meet any business needs. Its enterprise-ready features comprise of:

  • More Security and Privacy

Check Tardigrade Documentation for more.

What is ASP.NET Worker Service?

A worker service is a .NET project built using a template which supplies a few useful features that turn a regular console application into something more powerful. A worker service runs on top of the concept of a host, which maintains the lifetime of the application. The host also makes available some familiar features, such as dependency injection, logging, and configuration.

Worker services will generally be long-running services, performing some regularly occurring workload.

Referenced: https://www.stevejgordon.co.uk/what-are-dotnet-worker-services

Since we have some basic understanding of the tools that we’ll be working with, we can dive into the step by step on how we can achieve the data backup process of Fluree database snapshots to Tardigrade decentralized cloud storage with .NET Core worker service.

What You’ll Need

It is assumed you have some experience with ASP.NET Core, but not a whole lot. Topics such as Dependency Injection in .NET Core are outside the scope of this article, so it is suggested you familiarize yourself with them before continuing.

* You can use Visual Studio Code or an editor of your choice as well. We’ll mainly need VS for the Worker Service template.

The ASP.NET Core Worker Service template provides a starting point for writing long running service apps. An app created from the Worker Service template specifies the Worker SDK in its project file:

Getting Started

To get started, simply fire up Visual Studio, and create a new project of the Worker Service type:

Image for post

Worker Service Template in Visual Studio

Specify a name for your project, and optionally, tick the “Enable Docker Support” box in the last step if you want your worker service to be Docker enabled, we are not looking at Docker support in this article.

After completing the wizard, you will have a basic worker comprised of a Program.cs and Worker.cs file, as well as an appsettings.json file containing the settings for your application.

In this article, I will not be going deeper into how worker services work, you can read more about it here.

Fluree API

Before we can upload the Fluree snapshot to Tardigrade we need to understand how Fluree creates snapshots, how we can download, and where the snapshot can be located. Fluree already has great documentation that covers setting up the Fluree server on your local machine for different types of operating systems. You can find the documentation on installation here.

Fluree has a lot of REST API that performs different operations on the Fluree Server, But in this article, we are focusing more on the snapshot backup API.

Image for post
Snapshot documentation from the Fluree documentation

The snapshot endpoint gives us the ability to create a snapshot of any given database at any time.

Creating Fluree Database Snapshot.

The code snippet below handles the creation of snapshot from the Fluree API, It returns the snapshot file name that was created after a successful operation. This will also be hooked up with our worker service as we proceed.

We’ll be creating a background service using ASP.NET Core Worker services to run at an interval. This worker services will run automatically at the interval given to back up the snapshot into a folder and later proceed to upload the snapshot to Tardigrade. Let’s talk about the illustration of how this process is done.

Now that we have successfully created our worker service project, let’s create a class named BackupService.csthat will handle the functions that will help us finally, push our created snapshot to the Tardigrade Decentralized cloud.

I will be sharing some code snippets along with my illustration.

Let’s create the interface that’s going to expose all the method in our BackupService.cs class.

We’ve talked about the CreateSnapshot method, which handles the creation of a database snapshot from the Fluree server. We have also created a GetFlureeSnapshot method that will be responsible for getting the snapshot that was created by the CreateSnapshot method, this method has a return type of FileObject, which retrieve the snapshot File in a byte array and the snapshot file name.

The code snippet above searches through the Fluree snapshot folder to retrieve the snapshot file by the filename given.

Since we have been able to create and retrieve snapshots from the Fluree server successfully, the next step is to start the backup/upload process to the Tardigrade decentralized cloud storage.

Uploading our snapshots to Tardigrade Decentralized Cloud Storage.

I’ve created the UploadToCloud method in BackupService.csto handle the upload task for us, let’s take a deep look at it.

Before we can upload or interact with the Tardigrade Cloud Storage we need the uplink.NET SDK.

Uplink.NET SDK is available on Nuget, comprise of so many features that we can use to interact with the Tardigrade cloud, it has a lot of built-in methods that handle virtually everything we want to achieve or do with Tardigrade Cloud such as the creation of bucket, the listing of buckets, uploading of files and lots more. I recommend you check the Uplink.NET SDK documentation here.

Let’s quickly install the uplink.NET SDK in our WorkerService Project by typing this command on our Package Manager Console in your visual studio.

Install-Package uplink.NET -Version 2.2.2

After installation, we can start interacting with Tardigrade Cloud Storage via the SDK. In this article, we’ll be performing some operations such as creating a bucket, list buckets, uploading files, and also listing files.

Let’s start by initializing the Uplink Sdk to make its methods available for us to use in BackupService.cs . Before we can use the uplink.NET SDK we need to get the Tardigrade Satellite Address and Port, ApiKey, and Secret key.

To get the Tardigrade ApiKey and Satellite address and port, visit the Tardigrade website, the secret key is a key that Tardigrade use to encrypt your file while uploading files and the same secret key must be provided when retrieving the file too.

In the code snippet above, we initialized the uplink.Net SDK inside the UploadToCloud method, because that’s the method that’s responsible for uploading the Fluree snapshot to Tardigrade.

The next step is to list and create a bucket:

Now that we’ve created a bucket named flureebucket. That’s the bucket that will be storing the snapshot every time our background Worker Service runs.

The next step is to finally, upload the retrieved snapshot from Fluree Server to the flureebucket we created in the Tardigrad decentralized storage.

The code snippet above retrieves the snapshot file and performs the upload task, it also gives us the flexibility to track the upload progress and when the upload is completed which we can subscribe to, and make some further decisions.

If the upload task is successful, we need to verify that the snapshot has been uploaded to Tardigrade decentralized cloud storage via the code snippet below.

Finally, before we invoke theUploadToCloud method in the Worker Service, we need to register the BackupService.cs dependency injection to make it available to the Worker Service process at runtime.

To register the necessary classes, we need to edit the Program.cs file and paste the following code snippet.

After we have registered every dependency injection, we can now invoke the UploadToCloud method in the Worker Service. We need to edit the Worker.cs file and paste the code snippet below.

We can set the time interval for the process to run in the background. In the above code snippet, we set its timer to run at every 5 secs interval.

If all went well, you should see this screen when you run your Worker Service project.

Image for post

Upload and retrieve the list of files

Conclusion

Fluree & Tardigrade together can create a Highly Secure, Scalable, Effective Solution for modern Data Ecosystems, both for B2B and B2C scenarios.

CloudBloq, a Tools & Technology Startup and Fluree and Tardigrade Partner; provides Managed Data Platform on Tardigrade and Fluree technologies, with “API First” design. CloudBloq also offers System Integrator services on Tardigrade and Fluree based technologies.

Awesome! please share it with anyone you think could use this information thanks for reading.