Data abstraction interfaces

From HiHAT
Jump to navigation Jump to search

The purpose of this page is to record input on requirements, offerings, and proposals for data abstraction interfaces, as part of the overall HiHAT community effort. For a WIP picture of memory abstractions, see here.

  • Layers
    • User facing
      • Examples: OpenMP, MPI, Sidre, SICM, CHAI, Umpire, ...
    • Retargetable plumbing
      • Examples: HiHAT User and Common Layer APIs
    • User-level implementations
      • Examples: memkind, libpmem, cnmem, tcmalloc, jemalloc, cudaMalloc, SCR
    • Kernel-level support, and interfaces to that support
      • Examples: OS.SICM, mmap, libnuma
  • Requirements
    • Kinds of memory objects
      • Regular - simple contiguous arrays, multi-D arrays with stride, block (ref HDF5, MPI data types)
      • Irregular
    • Representation
      • Declarative approach
        • Semantic properties: accessibility (R/W), size
        • Performance properties: affinity to resources, memory kind, layout
        • Mutable state: pinned, cleared, valid, materialized
      • Layout
        • Distinct from logical handle
        • May or may not be changeable without creating a new instance
      • Hierarchy
        • Provide a formulaic prescription for how data could be partitioned, without overdecomposing it a priori
    • Asynchrony
      • Logical handle is available before async operations complete
      • Async operations: virtual alloc, physical materialization, affinitization, pinning, clearing
    • Allocation
      • Pluggable allocators, e.g. unified, per-thread, same-size
    • Operations
      • Conflicts: perfect match, partial match, overlapping ranges without conflicts ({.x} and {.y,.z})
      • Access: slices, projections