Chapter 17 of 34

Own AreaModel: two million rows without an array

The table only asks for visible cells – so a model can compute values on demand.

⏱ 6 min
▶ Video episode 017
git tag ep-017
needs @guiexpert/table@1.1.85

After this chapter you can

extend AreaModel
make scrolling O(1) with getConstantRowHeight()
load data page by page from a server

The table only asks for visible cells

chapter 11 (TableModel & areas) showed it: the table asks getRowCount() and getValueAt() for the cells on screen. So a model doesn't need an array – it can compute each value when asked.

import { AreaModel } from "@guiexpert/table";

const COUNTRIES = ["Germany", "Austria", "Switzerland", "France", "Italy", "Spain", "Poland", "Czechia"];
const STATUS = ["open", "paid", "shipped", "cancelled"];
const DAY = 86_400_000, START = Date.UTC(2020, 0, 1);

// cheap, deterministic "random" number per row and column (same row → same value, every time)
function hash(row: number, col: number): number {
  let h = Math.imul(row ^ (col << 24), 0x9e3779b1);
  h = Math.imul(h ^ (h >>> 16), 0x85ebca6b);
  return (h ^ (h >>> 13)) >>> 0;
}

export class TwoMillionRows extends AreaModel {
  constructor(private readonly rows = 2_000_000) {
    super();
    this.areaIdent = "body";
  }
  override getRowCount(): number { return this.rows; }
  override getRowHeight(_rowIndex: number): number { return 34; }
  override getConstantRowHeight(): number { return 34; }   // all rows equally high → O(1) while scrolling

  override getValueAt(row: number, col: number): any {      // called only for visible cells
    switch (col) {
      case 0: return row + 1;                                                          // ID
      case 1: return new Date(START + (row % 2_400) * DAY).toISOString().slice(0, 10); // date
      case 2: return COUNTRIES[hash(row, 2) % COUNTRIES.length];                       // country
      case 3: return (hash(row, 3) % 100_000) / 100;                                   // amount
      case 4: return STATUS[hash(row, 4) % STATUS.length];                             // status
    }
    return "";
  }
}

Extend AreaModel and override what you need – the base class has sensible defaults for the rest.
getValueAt() is called for about 20 rows × 5 columns per frame, not for 2 million rows.
AreaModelIf.getConstantRowHeight() tells the table that all rows are equally high: it finds the first visible row by division instead of adding up heights – O(1) instead of O(n) per scroll frame.

Use it

import {
  ColumnDef, EventAdapter, SimpleDomService, TableFactory, TableOptions, TableScope, allRight, bodyLeft, px120, px160
} from "@guiexpert/table";

const tableModel = TableFactory.createTableModel({
  bodyAreaModel: new TwoMillionRows(rowCount),
  columnDefs: [
    new ColumnDef("id", "ID", px120, allRight),
    new ColumnDef("date", "Date", px120),
    new ColumnDef("country", "Country", px160, bodyLeft),
    new ColumnDef("amount", "Amount", px120, allRight),
    new ColumnDef("status", "Status", px120)
  ]
});
const host = document.querySelector<HTMLDivElement>(".table-div")!;
const scope = new TableScope(host, tableModel, new SimpleDomService(), new TableOptions(), new EventAdapter())
  .firstInit();
const api = scope.getApi();

Screenshot: 2,000,000 rows, no array — Jump to the end: row 2,000,000 renders as fast as row 1. The toolbar shows frames per second while you scroll.

Data from a server, page by page

The same pattern works for remote data: getRowCount() returns the total from the server, getValueAt() answers from a cache and requests missing pages. When a page arrives, repaint:

import type { Order } from "./data/orders";

/** Sketch: rows come from a server, 100 at a time. */
export class PagedOrders extends AreaModel {
  private readonly pages = new Map<number, Order[]>();
  private readonly loading = new Set<number>();

  constructor(private readonly total: number, private readonly onLoaded: () => void) {
    super();
    this.areaIdent = "body";
  }
  override getRowCount(): number { return this.total; }
  override getRowHeight(_rowIndex: number): number { return 34; }
  override getConstantRowHeight(): number { return 34; }

  override getValueAt(row: number, col: number): any {
    const page = Math.floor(row / 100);
    const rows = this.pages.get(page);
    if (!rows) {
      this.load(page);
      return "…";                                          // placeholder until the page has arrived
    }
    return Object.values(rows[row % 100])[col];
  }

  private async load(page: number): Promise<void> {
    if (this.loading.has(page)) return;
    this.loading.add(page);
    const res = await fetch(`/api/orders?offset=${page * 100}&limit=100`);
    this.pages.set(page, await res.json());
    this.onLoaded();                                       // e.g. () => tableApi.repaint()
  }
}

Sorting and filtering then belong on the server, too – the model cannot sort rows it has never seen.

The showcase Large dataset uses the same idea with 2 million rows and renderers; Render speed measures it.

Wide tables: chapter 18 (Fixed columns).

Own AreaModel: two million rows without an array

The table only asks for visible cells

Use it

Data from a server, page by page

More

Next

API used in this chapter

Showcases