Skip to content

CBG-4806: move conflict detection to common function #7683

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Aug 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions db/crud.go
Original file line number Diff line number Diff line change
Expand Up @@ -1267,6 +1267,7 @@ func (db *DatabaseCollectionWithUser) PutExistingCurrentVersion(ctx context.Cont
revTreeConflictChecked := false
var parent string
var currentRevIndex int
var isConflict bool

// Conflict check here
// if doc has no HLV defined this is a new doc we haven't seen before, skip conflict check
Expand All @@ -1277,11 +1278,12 @@ func (db *DatabaseCollectionWithUser) PutExistingCurrentVersion(ctx context.Cont
return nil, nil, false, nil, addNewerVersionsErr
}
} else {
if doc.HLV.isDominating(newDocHLV) {
isConflict, err = IsInConflict(ctx, doc.HLV, newDocHLV)
if err != nil && errors.Is(err, ErrNoNewVersionsToAdd) {
base.DebugfCtx(ctx, base.KeyCRUD, "PutExistingCurrentVersion(%q): No new versions to add. existing: %#v new:%#v", base.UD(newDoc.ID), doc.HLV, newDocHLV)
return nil, nil, false, nil, base.ErrUpdateCancel // No new revisions to add
}
if newDocHLV.isDominating(doc.HLV) {
if !isConflict {
// update hlv for all newer incoming source version pairs
addNewerVersionsErr := doc.HLV.AddNewerVersions(newDocHLV)
if addNewerVersionsErr != nil {
Expand Down
50 changes: 50 additions & 0 deletions db/hybrid_logical_vector.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@
package db

import (
"context"
"encoding/base64"
"errors"
"fmt"
"maps"
"sort"
Expand Down Expand Up @@ -689,3 +691,51 @@ func (hlv *HybridLogicalVector) UnmarshalJSON(inputjson []byte) error {
func (hlv HybridLogicalVector) GoString() string {
return fmt.Sprintf("HybridLogicalVector{CurrentVersionCAS:%d, SourceID:%s, Version:%d, PreviousVersions:%#+v, MergeVersions:%#+v}", hlv.CurrentVersionCAS, hlv.SourceID, hlv.Version, hlv.PreviousVersions, hlv.MergeVersions)
}

// ErrNoNewVersionsToAdd will be thrown when there are no new versions from incoming HLV to be added to HLV that is local
var ErrNoNewVersionsToAdd = errors.New("no new versions to add to HLV")

// IsInConflict is used to identify if two HLV's are in conflict or not. Will return boolean to indicate if in conflict
// or not and will error for the following cases:
// - Local HLV dominates incoming HLV (meaning local version is a newer version that the incoming one)
// - Local CV matches incoming CV, so no new versions to add
func IsInConflict(ctx context.Context, localHLV, incomingHLV *HybridLogicalVector) (bool, error) {
incomingCV := incomingHLV.ExtractCurrentVersionFromHLV()
localCV := localHLV.ExtractCurrentVersionFromHLV()

// check if incoming CV and local CV are the same. This is needed here given that if both CV's are the same the
// below check to check local revision is newer than incoming revision will pass given the check and will add the
// incoming versions even if CVs are the same
if localCV.Equal(*incomingCV) {
base.TracefCtx(ctx, base.KeyVV, "incoming CV %#+v is equal to local revision %#+v", incomingCV, localCV)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I generally like %#+v for logging, but I think %#v might be better for HLV and CV since it displays versions in decimal instead of a hexadecimal.

Can you check this? If this is true, can you file a ticket to create a ruleguard rule so that we can detect this for the future?

If not, disregard this comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It only seems to wrap the formatted bit into quotes.

Internally we handle the HLV in uint64 type so logging here for example should be fine although %#v seems to format a uint64 into hex.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's worth filing a ticket to evaluate the places we log this and make sure we don't log in hex, it just makes everything harder to handle. But I can assert https://github.com/couchbaselabs/sg_dev_tools/pull/37 will handle this too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return false, ErrNoNewVersionsToAdd
}

// standard no conflict case. In the simple case, this happens when:
// - Client A writes document 1@srcA
// - Client B pulls document 1@srcA from Client A
// - Client A writes document 2@srcA
// - Client B pulls document 2@srcA from Client A
if incomingHLV.DominatesSource(*localCV) {
return false, nil
}

// local revision is newer than incoming revision. Common case:
// - Client A writes document 1@srcA
// - Client A pushes to Client B as 1@srcA
// - Client A pulls document 1@srcA from Client B
//
// NOTE: without P2P replication, this should not be the case and we would not get this revision, since Client A
// would respond to a Client B changes message that Client A does not need this revision
if localHLV.DominatesSource(*incomingCV) {
return false, ErrNoNewVersionsToAdd
}
// Check if conflict has been previously resolved.
// - If merge versions are empty, then it has not be resolved.
// - If merge versions do not match, then it has not been resolved.
if len(incomingHLV.MergeVersions) != 0 && len(localHLV.MergeVersions) != 0 && maps.Equal(incomingHLV.MergeVersions, localHLV.MergeVersions) {
base.DebugfCtx(ctx, base.KeyVV, "merge versions match between local HLV %#v and incoming HLV %#v, conflict previously resolved", localHLV, incomingCV)
return false, nil
}
return true, nil
}
49 changes: 49 additions & 0 deletions db/hybrid_logical_vector_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1413,3 +1413,52 @@ func TestAddVersion(t *testing.T) {
})
}
}

func TestIsInConflict(t *testing.T) {
testCases := []struct {
name string
localHLV string
incomingHLV string
expectedInConflict bool
expectedError bool
}{
{
name: "CV equal",
localHLV: "111@abc;123@def",
incomingHLV: "111@abc;123@ghi",
expectedError: true,
},
{
name: "no conflict case",
localHLV: "111@abc;123@def",
incomingHLV: "112@abc;123@ghi",
},
{
name: "local revision is newer",
localHLV: "111@abc;123@def",
incomingHLV: "100@abc;123@ghi",
expectedError: true,
},
{
name: "merge versions match",
localHLV: "130@abc,123@def,100@ghi;50@jkl",
incomingHLV: "150@mno,123@def,100@ghi;50@jkl",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
localHLV, _, err := extractHLVFromBlipString(tc.localHLV)
require.NoError(t, err)
incomingHLV, _, err := extractHLVFromBlipString(tc.incomingHLV)
require.NoError(t, err)

inConflict, err := IsInConflict(t.Context(), localHLV, incomingHLV)
if tc.expectedError {
require.Error(t, err)
} else {
require.NoError(t, err)
}
require.Equal(t, tc.expectedInConflict, inConflict)
})
}
}
28 changes: 4 additions & 24 deletions rest/utilities_testing_blip_client.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ import (
"encoding/base64"
"fmt"
"iter"
"maps"
"net/http"
"slices"
"strconv"
Expand Down Expand Up @@ -325,35 +324,16 @@ func (cd *clientDoc) _hasConflict(t testing.TB, incomingHLV *db.HybridLogicalVec
incomingCV := incomingHLV.ExtractCurrentVersionFromHLV()
localCV := localHLV.ExtractCurrentVersionFromHLV()
// safety check - ensure SG is not sending a rev that we already had - ensures changes feed messaging is working correctly to prevent
// this check is also performed in the function below but we should keep this here for extra context on FailNow call
if localCV.Equal(*incomingCV) {
require.FailNow(t, fmt.Sprintf("incoming CV %#+v is equal to local revision %#+v - this should've been filtered via changes response before ending up as a rev. This is only true if there is a single replication occurring, two simultaneous replications (e.g. P2P) could cause this. If there are multiple replications, modify code.", incomingCV, latestRev))
}
// standard no conflict case. In the simple case, this happens when:
// - SG writes document 1@cbs1
// - CBL pulls document 1@cbs1
// - SG writes document 2@cbs1
if incomingHLV.DominatesSource(*localCV) {
return false
}

// local revision is newer than incoming revision. Common case:
// - CBL writes document 1@cbl1
// - CBL pushes to SG as 1@cbl1
// - CBL pulls document 1@cbl1
//
// NOTE: without P2P replication, this should not be the case and we would not get this revision, since CBL
// would respond to a SG changes message that CBL does not need this revision
if localHLV.DominatesSource(*incomingCV) {
inConflict, err := db.IsInConflict(t.Context(), &localHLV, incomingHLV)
if err != nil {
require.FailNow(t, fmt.Sprintf("incoming CV %#+v has lower version than the local revision %#+v - this should've been filtered via changes response before ending up as a rev. blip tester would reply that to Sync Gateway that it doesn't need this revision", incomingCV, localHLV))
Copy link
Preview

Copilot AI Aug 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error handling logic is incorrect. The FailNow call with the 'lower version' message is triggered when any error occurs from IsInConflict, but according to the new function, errors can occur for multiple reasons including equal CVs or newer local versions. The error message should be updated to reflect the actual error or the error should be properly differentiated.

Suggested change
require.FailNow(t, fmt.Sprintf("incoming CV %#+v has lower version than the local revision %#+v - this should've been filtered via changes response before ending up as a rev. blip tester would reply that to Sync Gateway that it doesn't need this revision", incomingCV, localHLV))
require.FailNow(t, fmt.Sprintf("IsInConflict error: %v (incoming CV %#+v, local revision %#+v)", err, incomingCV, localHLV))

Copilot uses AI. Check for mistakes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will leave this as is given the extra context in original message.

return false
}
// Check if conflict has been previously resolved.
// - If merge versions are empty, then it has not be resolved.
// - If merge versions do not match, then it has not been resolved.
if len(incomingHLV.MergeVersions) != 0 && len(localHLV.MergeVersions) != 0 && maps.Equal(incomingHLV.MergeVersions, localHLV.MergeVersions) {
return false
}
return true
return inConflict
}

func (btcc *BlipTesterCollectionClient) _resolveConflict(incomingHLV *db.HybridLogicalVector, incomingBody []byte, localDoc *clientDocRev) (body []byte, hlv db.HybridLogicalVector) {
Expand Down
Loading