4

How can we synchronize accessing a common resource using synchronization in Java when my java application is deployed on multiple instances behind a load balancer ?

Because, as far as I know, synchronization works only under one JVM. But when we deploy the same Java application on multiple instances to handle the load, then how can we provide a synchronization mechanism?

For example:- There is a HDFS file to which the java application either appends/edits the contents of that HDFS file. When I deploy my java application on multiple instances, then how can I make sure that only one request from Java application accesses that HDFS file ?

Improve this question
3
  • 1
    You need some sort of distributed locking service. Commented May 11, 2019 at 8:32
  • AFAIK, HDFS files cannot be edited, only appended. Appending does not require locks. Commented May 11, 2019 at 9:58
  • @AlexeiKaigorodov I'm talking about locks when Java application instances try to access the same HDFS file. Commented May 12, 2019 at 7:59

2 Answers 2

Reset to default
2

Short answer - You can't do it without introducing a lot of complexities in your setup.

While technically you can use something like a distributed lock that's available on Zookeeper. I wouldn't really recommend them. It's kinda hard to reason with them at scale and there's also additional complexity in terms of Zookeeper Operations itself.

Regarding the example you posted, isn't that why systems like HBase was built? Model your data into a Key->[Multiple Columns] format. You can then read / write data on HBase and it would behind the scenes do the heavy lifting of editing / managing multiple files for you.

On the other hand, if you can model your change that you want to do on your file as an event you can then build your system on the principles of Event Driven Architecture.

You can read more about this on

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

I can recommend you using the distributed lock mechanism provided by redis. It works the same as standard lock or mutex but works in the context of a distributed system. The application instance that creates the lock will block the access to the resource to do the changes, then it releases the lock to allow other instances get access to the resource.

We already use this solution in production to protect the access to some critical resources that do not provide synchronization and consistency natively.

Here is the link for redis distributed lock: Distributed locks with Redis

I believe there are other solutions providing the same feature. Redis is very lightweight, scalable and easy to integrate.

Improve this answer

3 Comments

Incase we go for redis to do the locking mechanism, even redis itself will be having multiple instances deployed, right? In that case, how will it work ?
yes redis will be replicated too. it is however responsible for synchronizing the replica set and for guaranteeing that the distributed lock is visible across all the replicas.
Sounds cool!. Thanks. Can you share all the available distributed locking mechanism providers in the market?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.