Mutex for Bleve Write/Read Operation

Game Design
1

Hi,

Supposed I have a system for asynchronous PvP matchmaking using Bleve which has three functionalities.

  • The first one is for periodically cleaning up data in Bleve if the data in Bleve is outdated.
  • The second one is for reading the data from Bleve.
  • The third one is for writing data to Bleve from Nakama Storage.

Based on this functionality, the cleaning up functionality can only be executed by 1 instance, while the read and write functionality can be executed by multiple instances concurrently.

To ensure the data integrity, I know a mutex should be applied to all 3 functionalities. However, I’m not sure how to implement the mutex correctly. I’m afraid that by using mutex, the read functionality will be blocked. This is concerning since the read functionality needs to respond quickly.

Any thoughts on what kind of mutex that I should use? Also, where should I put the mutex? Should I put it to all 3 of the functionalities?

Here is my simple code for the read and write functionality, I haven’t implemented it in nakama yet.

 func Read() {
	index, err := bleve.Open("player.bleve")
	defer index.Close()
	if err != nil {
		log.Fatal(err)
		return
	}

	query := bleve.NewQueryStringQuery("*")
	req := bleve.NewSearchRequest(query)

	res, err := index.Search(req)
	if err != nil {
		log.Fatal(err)
		return
	}
	fmt.Println(res)
}

func Write(player *Player) {
	index, err := bleve.Open("player.bleve")
	defer index.Close()
	if err != nil {
		log.Fatal(err)
		return
	}

	index.Index(player.ID, player)
}
2

@adhipradhana The use of Bleve for this kind of asynchronous matchmaker logic is a good one. We’ve done it before with game teams on multiple projects. I have a few questions that will help frame how you handle concurrent access and how the data in the search index is managed:

  1. What are the fields you want to index in Bleve?
  2. What do you think a typical query to filter records on will look like?
  3. How do you plan to initialize the index at server start up?
  4. How do you plan to purge records that are stale or have changed over time?
3

@novabyte Thank you for answering my question,

  1. What are the fields you want to index in Bleve?

I try to index multiple numeric fields and multiple string fields in Bleve. A typical record in Go would look like below. Do you think that I should index the string fields if I only need to perform query on the numeric fields?

type Record struct {
	ID            string
	NumericField1 int
	NumericField2 int
	StringField1  string
	StringField2  string
	ExpiredTime   time.Time
}
  1. What do you think a typical query to filter records on will look like?

I try to execute a ranged numerical query on the NumericFields1 and NumericFields2 to filter the records. A sample query might look like below.

query := bleve.NewQueryStringQuery("+NumericFields1:>=10 +NumericFields2:<5")
  1. How do you plan to initialize the index at server start up?

Since I’m trying to avoid moving all of the data from the Nakama Storage into the Bleve search engine, I’m planning to only create the index mapping during the server startup.

For filling the record into Bleve, I plan to implement RPC which index only one record at a time from Nakama Storage. Thus, the indexing process would be light. The RPC would not be called in batch.

  1. How do you plan to purge records that are stale or have changed over time?

To purge outdated record, I want to implement a goroutine that will be executed every certain duration (e.g. every 6 hours) by querying all of the records that exceed the ExpiredTime field. To update the record, I plan to execute the RPC that I have previously explained in point 3.

Additional question, The reason why I don’t use Nakama’s matchmaking system is because I also want to query offline users during the matchmaking process. Do you think it would be better to perform the ranged numerical query on the Bleve or directly on the Nakama Storage?