Overview

The kinds of memory are defined by operating system memory policies that have been applied to virtual address ranges. Memory characteristics supported by memkind without user extension include control of NUMA and page size features. The jemalloc non-standard interface has been extended to enable specialized arenas to make requests for virtual memory from the operating system through the memkind partition interface. Through the other memkind interfaces the user can control and extend memory partition features and allocate memory while selecting enabled features.
Memkind is hosted on GitHub

Downloads

Documentation

Architecture [PDF] memkind API hbwmalloc API

Sources

 $ git clone https://github.com/memkind/memkind.git 

Tarball (master) ZIP (master)

Binaries

RPMs for misc OSes

FAQ

Is memkind package part of any official Linux Distribution?

Memkind package is currently available as part of Fedora 22, Fedora 23 and Fedora 24. Also, it is available in EPEL repository. Fedora package database

Blog

Release 1.6.0 is available

I would like to announce memkind release 1.6.0. Changes since 1.5.0:

  • Introduced new policy HBW_POLICY_BIND_ALL.
  • Introduced MEMKIND_HBW_ALL, MEMKIND_HBW_ALL_HUGETLB and MEMKIND_REGULAR kinds.
  • Fixed hbw_posix_memalign_psize() return value in case of unsupported policy and page size combination.
  • Documented supported policy and page size combination for hbw_posix_memalign_psize().
  • Merged libmemkind.a and libjemalloc_pic.a into a single static library.
  • Fixed OOM during tcache acquisition.
  • Tuned huge pages configuration in memkind tests.
  • Added stress tests for allocations with large random sizes.
  • Added stress tests for allocations with random kinds.
  • Extended policy and NUMA node binding tests.
  • Fixed several Travis build problems.
  • Removed gb_page_tests_preferred_policy tests.

Sources are available on: https://github.com/memkind/memkind/releases/tag/v1.6.0

Best Regards,

Artur Koziej

Release 1.5.0 is available

I would like to announce memkind release 1.5.0. Changes since 1.4.0:

  • Intel(R) TBB scalable_allocator can be used as heap management alternative to jemalloc (requires installed Intel (R) TBB package and environment variable MEMKIND_HEAP_MANAGER=TBB)
  • Updated internally used jemalloc to upstream version 4.3.1 (previously modified 3.5.1)
  • Introduced new environment option MEMKIND_HOG_MEMORY. Setting it to "1" will prevent memkind from releasing memory to OS.
  • Introduced support for KNM processor from Intel(R) Xeon Phi(TM) family
  • General cleanup of build scripts (e.g. fixed issues found on Ubuntu, introduced simple build.sh script)
  • Deprecated support for gigabyte-pages
  • Removed number of previously deprecated symbols

Sources are available on: https://github.com/memkind/memkind/releases/tag/v1.5.0

Best Regards,

Krzysztof Kulakowski

Release 1.4.0 is available

I would like to announce memkind release 1.4.0. Changes since 1.3.0:

  • Introduced hbw_verify_memory_region() for verifying if provided memory region fully falls into high-bandwidth memory (for details please take a look at man/hbwmalloc.3).
  • New API for creating kinds using set of predefined attributes: memkind_create_kind(), memkind_destroy_kind() (for details please take a look at man/memkind.3).
  • Fixed issue in autohbw where it was reporting limit error regardless of value provided.
  • Added logging on error paths.
  • Test base fixes and improvements.

Sources are available on: https://github.com/memkind/memkind/releases/tag/v1.4.0

Best Regards,

Krzysztof Kulakowski

Release 1.3.0 is available

I would like to announce memkind release 1.3.0. Changes since 1.2.0:

  • Introduced logging mechanism (for details please take a look at man/memkind.3).
  • Deprecated symbols: memkind_finalize(), memkind_get_num_kind(), memkind_get_kind_by_partition(), memkind_get_kind_by_name(), memkind_partition_mmap(), memkind_get_size(), MEMKIND_ERROR_MEMALIGN, MEMKIND_ERROR_MALLCTL, MEMKIND_ERROR_GETCPU, MEMKIND_ERROR_PMTT, MEMKIND_ERROR_TIEDISTANCE, MEMKIND_ERROR_ALIGNMENT, MEMKIND_ERROR_MALLOCX, MEMKIND_ERROR_REPNAME, MEMKIND_ERROR_PTHREAD, MEMKIND_ERROR_BADPOLICY, MEMKIND_ERROR_REPPOLICY.
  • Added integration with hwloc (turned on with --with-hwloc).
  • Cleanup of symbols exposed by libmemkind.so (e.g. no longer exposing libnuma and jemalloc symbols).
  • AutoHBW files have been moved to memkind/autohbw directory, code has been refactored and tests have been added to appropriate scenarios.
  • Library is now built with flags improving security (can be turned off with --disable-secure configure time option).
  • Changed configuration of jemalloc to turn off unused features.

Sources are available on: https://github.com/memkind/memkind/releases/tag/v1.3.0

Best Regards,

Krzysztof Kulakowski

Release 1.2.0 is available

I would like to announce memkind release 1.2.0. Changes since 1.1.1:

  • memkind_create() and memkind_ops have been deprecated (moved to memkind_deprecated.h).
  • Deprecated the headers from memkind/internal and added compile-time warnings to them (plan is to remove those from rpm packages).
  • autohbw files have been moved to memkind/autohbw directory.
  • API decorators (memkind_malloc_pre, memkind_malloc_post etc.) now need to be enabled with configure-time option --enable-decorators.
  • Allocation time optimizations with up to 20% improvement.

Sources are available on: https://github.com/memkind/memkind/releases/tag/v1.2.0

Best Regards,

Krzysztof Kulakowski

RPMs based on dev branch are here!

Memkind source code is tracked with two git branches:

  • master - stable, production-quality source code. Here memkind releases are done and tagged
  • dev - development branch which contains not-yet-finished features, new proposals, etc.

So far, prebuilt RPMs were only available for master branch and latest greatest release. Today, we introduce RPMs for latest greatest 'dev' branch, in case one would like to go ahead and try new functionality which has not yet been blessed with official release. Link to RPMs storage, as usual in Download|Binaries section.

Happy hacking!

Release 1.1.1 is available

I would like to announce memkind release 1.1.1. Changes since 1.1.0:

  • Performance improvement for memkind_free() in scenario where NULL was passed as kind (reduced by 60%).
  • Introduced integration with Travis CI.
  • Fixed issue where memory returned from calloc was not filled with zeroes when using memkind_pmem kinds.
  • Fixed issue where interleave kinds was failing on systems without Transparent Huge Pages module configured.
  • Resolved several issues that was causing compilation errors on some systems.
  • test/test.sh is now able to detect system configuration, and run only those tests which requirements are meet.
  • Added gtest to repo to avoid downloading it during build proecess.

Sources are available on: https://github.com/memkind/memkind/releases/tag/v1.1.1

Best Regards,

Krzysztof Kulakowski

Travis CI is here!

Travis-CI for memkind has been enabled and integrated. Here you can find memkind's yaml file that is used by Travis to fire-up CI builds and tests. Please note that CI testing is disabled at the moment, as 'make test' needs to be tuned for Travis infrastructure.

In case of any issues, do not hesitate to contact us!

Release 1.1.0 is available

I would like to announce memkind release 1.1.0. Changes since 1.0.0:

  • Implemented algorithm for detecting high-bandwidth memory on Intel Xeon Phi X200 without parsing PMTT.
  • PMTT-parsing service has been removed.
  • memkind struct definition from memkind.h has been replaced by forward declaration. Definition has been moved to /internal/memkind_private.h and shouldn't be considered as part of library interface anymore.
  • Performance improvment for hugetlb kinds (check_available() is no longer parsing sysfs every time).
  • Performance fix by removing --enable-safe from jemmaloc build flags.
  • New tests for validating high-bandwidth memory detection has been added.
  • AFTS has been split into two groups: memkind-afts.ts and memkind-afts-ext.ts, based on amount of required memory (for details view test/README).

Sources are available on: https://github.com/memkind/memkind/releases/tag/v1.1.0

Best Regards,

Krzysztof Kulakowski

Release 1.0.0 is available

I would like to announce memkind release 1.0.0. Changes since 0.3.0 (beta):

  • hbwmalloc.h released as stable API.
  • Introduce API standards: STANDARD API, NON-STANDARD API, EXPERIMENTAL API.
  • Introduce memkind versioning API.
  • Fix type names in hbwmalloc API.
  • Change error codes to POSIX standard ones in hbwmalloc API.
  • Move API headers to include directory.
  • Move NON-STANDARD API, or EXPERIMENTAL API to include/memkind/internal directory.
  • New man page (memkind_interleave.3) describing interleave kind.
  • Significant documentation improvement.
  • Performance fix for jemalloc - significantly decrease of library initialization time.
  • Enforce 2MB alignment in jemalloc due to the Linux kernel bug (munmap() fails for huge pages, when the size is not aligned).
  • Move to systemd defined service (memkind.service), drop init.d script.
  • New rpm layout (memkind and memkind-devel).
  • Remove false dependency on OpenMP.
  • Extend test base for stress, longevity and initialization performance tests.
  • Fix memory leak in ctl_growk().
  • Rename namespace: hbwmalloc -> hbw.
  • Rename class: hbwmalloc_allocator -> allocator.

Sources are available on: https://github.com/memkind/memkind/releases/tag/v1.0.0

Best Regards,

Krzysztof Kulakowski

Release 0.3.0 (beta) is available

I would like to announce memkind release 0.3.0 (beta). Changes since 0.2.2:

  • Added two new kinds: MEMKIND_INTERLEAVE and MEMKIND_HBW_INTERLEAVE (for details please take a look at man/memkind.3)
  • Added support for file-backed memory heaps (for details please take a look at man/memkind_pmem.3)
  • Added autohbw library, that intercepts the standard heap allocations, which let use high-bandwidth memory without modifying existing codebase (for details please take a look at examples/autohbw_README)
  • jemalloc is now, statically linked, internal component of memkind RPM instead of rpm dependency.
  • Added memkind_allocated example which demonstrates usage of memkind with alignas() alignment specifier introduced in C++11 (for details please take a look at examples/README)
  • Added support for using thread local storage to improve performance in multithreaded applications (enabled by configure time option: --enable-tls)
  • Added possibility to set number of jemalloc arenas per kind by configure time parameter or environmental variable (for details please take a look at MEMKIND_ARENA_NUM_PER_KIND section of man/memkind.3).
  • Added decorators to the memkind allocation APIs. These are weak symbols (pre and post for each API) which can modify the input and output of each of the calls.
  • Significantly extended test base (new groups of tests: performance, multithreaded, PMTT, negative, stress).

Sources are available on: https://github.com/memkind/memkind/releases/tag/v0.3.0

Best Regards,

Krzysztof Kulakowski

Contact us

The Memkind Library team at Intel Corporation, led by Lukasz Anaczkowski, created this project on GitHub to encourage community development. Your participation, feedback, and questions are welcome. You can:

  • Create an issue in our GitHub issues database. This is a good way to report problems or request new features.
  • Join our mail list to participate in discussions on memkind, including the design of th API and future directions.