Infinite query fetchNextPage modifies first page's data

I have a mutation where I append a comment to the first page (pages[0]). In my onMutate, I cancel any ongoing refetches (if we're not fetching a subsequent page) and then refetch in onSettled

Mutation:

    const createCommentMutation = useMutation({
    mutationFn: createComment,
    onMutate: async (variables) => {
        const { text, replyTo } = variables
        const cacheKey = replyTo ? ["postReplyComments", replyTo] : ["postComments", post._id]

        let wasFetchingFirstPage
        const query = queryClient.getQueryCache().find({ queryKey: cacheKey })
        if (query) {
            const observer = query.observers[0]
            const result = observer.getCurrentResult()

            wasFetchingFirstPage = result.isFetching && !result.isFetchingNextPage
        }

        wasFetchingFirstPage && queryClient.cancelQueries({ queryKey: cacheKey })

        const tempCommentId = uuidV4()
        const creatorData = {
            _id: userData._id,
            username: userData.username,
            profilePic: userData.profilePic,
        }
        const pendingComment = {
            _id: tempCommentId,
            createdAt: new Date().toISOString(),
        }

        queryClient.setQueryData(cacheKey, (prevData) => {
            return produce(prevData, (draft) => {
                const page = draft.pages[0]
                page.comments.unshift(pendingComment)

                if (wasFetchingFirstPage) page.pendingRefetch = true
            })
        })

        updateReply(null)
        textRef.current.value = ""

        return { pendingComment, cacheKey, wasFetchingFirstPage }
    },
    onSuccess: (data, variables, context) => {
        const { pendingComment, cacheKey } = context
        const { comment } = data
        // Optimistically replace comment in onsuccess
        queryClient.setQueryData(cacheKey, (prevData) => {
            return produce(prevData, (draft) => {
                const page = draft.pages[0]
                const pendingIndex = page.comments.findIndex(
                    (c) => c._id === pendingComment._id
                )
                const data = { ...comment, creator: pendingComment.creator }
                page.comments[pendingIndex] = data
            })
        })
    },
    onSettled: (_, __, ___, context) => {
        const { cacheKey, wasFetchingFirstPage } = context

        const refetchType = wasFetchingFirstPage ? "active" : "none"
        queryClient.invalidateQueries({ queryKey: cacheKey, refetchType })

        if (wasFetchingFirstPage) {
            queryClient.setQueryData(cacheKey, (prevData) => {
                return produce(prevData, (draft) => {
                    if (!prevData) return

                    const page = draft.pages[0]
                    page.pendingRefetch = false
                })
            })
        }
    },
})

The issue is when I call fetchNextPage() in my component, and whilst isFetchingNextPage is true, when I add a comment, and fetching completes, any comments added during this period are removed. My cursors are correct and pagination works as expected when I don't create any comments during this time.

I'm under the impression fetchNextPage() just appends a page to the pages array, however it is clearly modifying the first page. I can confirm invalidateQueries in onSettled is not the issue, as I've removed it and the issue persists. Also, I don't want to cancelQueries if we're fetching the next page.