Invalidating Custom Pagekeyeddatasource Makes Recycler View Jump

I am trying to implement an android paging library with custom PageKeyedDataSource, This data source will query the data from Database and insert Ads randomly on that page. I impl

Solution 1:

Every time invalidate() is called, the whole list will be considered invalid and built again in its whole, creating a new DataSource instance. It is actually the expected behaviour, but lets see a step by step sequence of what is happening under the hood to understand the problem:

1. A DataSource instance is created, and its loadInitial method is called, with zero items (As there is no data stored yet)
2. BoundaryCallback's onZeroItemsLoaded will be called, so first data will be fetched, stored and finally, it will invalidate the list, so it will be created again.
3. A new DataSource instance will be created, calling its loadInitial again, but this time, as there is already some data, it will retrieve those previously stored items.
4. User will scroll to the list's bottom, so a new page will be tried to be loaded from the DataSource by calling loadAfter, which will retrieve 0 items as there are no more items to be loaded.
5. So onItemAtEndLoaded in BoundaryCallback will be called, fetching the second page, storing the new items and finally invalidating the whole list again.
6. Again, a new DataSource will be created, calling once more its loadInitial, which will only retrieve the first page items.
7. After, once the loadAfter is called again, it will now be able to retrieve the new page items as they have just been added.
8. This will go on for each page.

The problem here can be identified at Step 6.

The thing is that every time we invalidate the DataSource, its loadInitial will only retrieve the first page items. Although having all the other pages items already stored, the new list will not know about their existence until their corresponding loadAfter is called. So after fetching a new page, storing their items and invalidating the list, there will be a moment in which the new list will only be composed by the first page items (as loadInitial will only retrieve those). This new list will be submitted to the Adapter, and so, the RecyclerView will only show the first page items, giving the impression it jumped up to the first item again. However, the reality is that all the other items have been removed as, in theory, they are no longer in the list. After that, once the user scrolls down, the corresponding loadAfter will be called, and the page items will be retrieved again from the stored ones, until a new page with no stored items yet is hit, making it invalidate the whole list again after storing the new items.

So, in order to avoid this, the trick is to make loadInitial not just always retrieve the first page items, but all the already loaded items. This way, once the page is invalidated and the new DataSource's loadInitial is called, the new list will no longer be composed by just the first page items, but by all the already loaded ones, so that they are not removed from the RecyclerView.

To do so, we could keep track of how many pages have already been loaded, so that we can tell to each new DataSources how many of them should be retrieved at loadInitial.

---

A simple solution would consist on creating a class to keep track of the current page:

classPageTracker {
    var currentPage = 0
}

Then, modify the custom DataSource to receive an instance of this class and update it:

classColorsDataSource(
    privateval pageTracker: PageTracker
    privateval colorsRepository: ColorsRepository
) : PageKeyedDataSource<Int, ColorEntity>() {

    overridefunloadInitial(
        params: LoadInitialParams<Int>,
        callback: LoadInitialCallback<Int, ColorEntity> 
    ) {
        //...val alreadyLoadedItems = (pageTracker.currentPage + 1) * params.requestedLoadSize
        val resultFromDB = colorsRepository.getColors(0, alreadyLoadedItems)
        callback.onResult(resultFromDB, null, pageTracker.currentPage + 1)
    }

    overridefunloadAfter(params: LoadParams<Int>, callback: LoadCallback<Int, ColorEntity>) {
        pageTracker.currentPage = params.key
        //...
    }

    //...
}

Finally, create an instance of PageTrackerand pass it to each new DataSource instance

dataSourceFactory = object : DataSource.Factory<Int, ColorEntity>() {

    val pageTracker = PageTracker()

    overridefuncreate(): DataSource<Int, ColorEntity> {
        dataSource =  ColorsDataSource(pageTracker, repository)
        return dataSource
    }
}

---

NOTE 1

It is important to note that if it is needed to refresh the whole list again (due to a pull-to-refresh action or anything else), PageTracker instance will be required to be updated back to currentPage = 0 before invalidating the list.

---

NOTE 2

It is also important to note that this approach is usually not required when using Room, as in this case we probably do not need to create our custom DataSource, but instead make the Dao directly return the DataSource.Factory directly from the query. Then, when we fetch new data due to BoundaryCallback calls and store the items, Room will automatically update our list with all the items.

Solution 2:

In DiffUtilCallback at areItemsTheSame compare ids instead of references:

overridefunareItemsTheSame(oldItem: ColorEntity, newItem: ColorEntity): Boolean 
              = oldItem.db_id == newItem.db_id

In this way the recyclerView will find previous position from ids instead of references.