Persistence in the Baseline Protocol Stack (BRI-1)

hello Devs!

I’m working on adding a Ipersistance code to core base line.

I get bri-1 example to work on my machine. Nevertheless i couldn’t figure how to add some custom base line to the code. Can you give me some indications ?

Thank you for your attention !

Hi @nabetse – I didn’t fully follow your question :grimacing:

There is some ongoing work related to the IPersistence interface in a Github fork over here. The effort is in the right direction for sure and could be used within BRI-1 to replace the in-memory reference implementation.

I hope this is helpful to you.

Hi @kthomas,

First, thank you for your reply.

I’ll try to clarify my question.

Lets assume that I have implemented a Ipersistance interface like the one you linked in your answer ( here)

What are the modifications that I have to make in order to get br-1/base-example stack to take it into account. I mean the images are pulled from the docker hub so I don’t know if the baseline protocol is pulled in a similar way or you can specify a custom package.

Hope I managed to clarify myself.

Hey @nabetse, this is a great question.

First, the BRI-1 comes as a dockerized setup built on top of Provide’s microservice architecture. The APIs of the Provide microsrevices implement interfaces of the Baseline Protocol. Have a look at the docker-compose.yml in bri-1/base-example here.

Second, the BRI-1 does not make use of any persistence provider instead it uses a simple in-memory mapping to store workflow records, see here.
Records get added to the mapping in ll. 444 here.

So, the modifications to the BRI-1 to use your persistence provider would be two-folded.
As a first step, if your provider depends on docker images/container, add them to the docker-compose.yml file.
Import or add your provider in the BRI-1 code.
After that delete the BRI-1 in-memory persistence code (e.g. l. 53, ll. 444) and modify the BRI-1 code to fit the needs and implementation of your provider.

I hope this helps.

1 Like

Hello @norym ,

Sorry for the late reply. Yes this helps a lot !
I’ll share my experience when i am done.

Again thanks for the answer!

ok @norym done some modification and ram persintence in order to work from here with bri-1.

I have some complains:

  • memory: nearly 6Gb of ram just to test. this is huge. I have to shut down everything on my laptop just to test. its not ok. I may be wrong but this should use some smaller images.
  • there is some typescript issues about void arguments getting to Promise . Easy to fix but my question is was it intended. changed it to null and no problems.

Sorry to bother you again @norym but would you consider a shared PR with bri-1 working ?

Again thanks for anyone getting this subject to work out !

hey @nabetse, great that you are working on the persistence integration. If possible share your implementation, etc. such that I or people in the community can have a look at your approach.

First, good catch on the argument handling for the RAM provider. Indeed, the RAM provider does not check for void, undefined inputs. I will update this.
However, the RAM provider is currently a proposal for a simple in-memory implementation of the persistence interface and is not merged to baseline repo yet.

I really like the idea of creating a shared PR for the integration of the RAM provider into BRI-1, since this would be the next best step.
Nevertheless, I suggest to wait until the PR including the RAM provider is eventually merged to baseline master in order to start with the integration because requirements for the provider my change or the implementation of the provider itself may change.
If you have fixes for the issues regarding the void, undefined inputs, let me know and I will merge them to the provider.

Second, I dont know if I understand your question regarding the ram usage during testing correctly. I assume your complain is about the docker image size of BRI-1 images, since the RAM provider itself does not depend on any image but indeed uses RAM to store the records. Maybe you can elaborate more on your issues.

Best

1 Like

Hello @norym,

Yes my rant was on docker images. But maybe it a necessity in this early stages.
I’ll share my implementation when i’m done !

Thanks for the reply

The Docker image requirement of 6GB is not unheard of. It’s running quite a few containers. We can perhaps look into the Nethermind client as it likely uses a lot of RAM.

The core Provide stack actually has a very small memory footprint.

Hello @kthomas,

Yes, you are correct since it runs 2 complete stacks 6GB it not that much.

I was testing a Ipersistence implementation for excel that required to work with the user offline - I don’t fully understand the requirement since Nethermind needs to be online to query the Ropsten Testnet but it’s ok.

So I needed to run that from windows or include a docker windows image with an excel install.

I have done it locally with no windows docker image.

After some tests (on my laptop - with only 8GB of ram and nvme hard drive) it seams that Nethermind client is responsible for unreliable behaviours (maybe my wifi connection too…). I mean sometimes npm test pass and others it doesn’t.

The code is here:

Review it if you can, I’ll be very helpful.

I’ve also done a sample video demo here.
Not the best quality, but it demonstrates the intended use.

Feel free to contact me for any suggestions or changes!
Again thank you for your attention!