qLibc
qhasharr.c File Reference

Static(array) hash-table implementation. More...

Go to the source code of this file.

Macros

#define COLLISION_MARK   (-1)
#define EXTBLOCK_MARK   (-2)

Functions

size_t qhasharr_calculate_memsize (int max)
 Get how much memory is needed for N slots.
qhasharr_t * qhasharr (void *memory, size_t memsize)
 Initialize a static hash table.
bool qhasharr_put (qhasharr_t *tbl, const char *name, const void *data, size_t datasize)
 qhasharr->put(): Put an object into this table.
bool qhasharr_putstr (qhasharr_t *tbl, const char *name, const char *data)
 qhasharr->putstr(): Put a string into this table
bool qhasharr_putstrf (qhasharr_t *tbl, const char *name, const char *format,...)
 qhasharr->putstrf(): Put a formatted string into this table.
bool qhasharr_put_by_obj (qhasharr_t *tbl, const void *name, size_t namesize, const void *data, size_t datasize)
 qhasharr->put_by_obj(): Put an object into this table by key object.
void * qhasharr_get (qhasharr_t *tbl, const char *name, size_t *datasize)
 qhasharr->get(): Get an object from this table
char * qhasharr_getstr (qhasharr_t *tbl, const char *name)
 qhasharr->getstr(): Finds an object with the given name and returns it as a string.
void * qhasharr_get_by_obj (qhasharr_t *tbl, const void *name, size_t namesize, size_t *datasize)
 qhasharr->get_by_obj(): Get an object from this table by key object.
bool qhasharr_remove (qhasharr_t *tbl, const char *name)
 qhasharr->remove(): Remove an object from this table.
bool qhasharr_remove_by_obj (qhasharr_t *tbl, const char *name, size_t namesize)
 qhasharr->remove_by_obj(): Remove an object from this table by key object
bool qhasharr_remove_by_idx (qhasharr_t *tbl, int idx)
 qhasharr->remove_by_idx(): Remove an object from this table by index number.
bool qhasharr_getnext (qhasharr_t *tbl, qhasharr_obj_t *obj, int *idx)
 qhasharr->getnext(): Get next element.
int qhasharr_size (qhasharr_t *tbl, int *maxslots, int *usedslots)
 qhasharr->size(): Returns the number of objects in this table.
void qhasharr_clear (qhasharr_t *tbl)
 qhasharr->clear(): Clears this table so that it contains no keys.
bool qhasharr_debug (qhasharr_t *tbl, FILE *out)
 qhasharr->debug(): Print the hash table for debugging purposes.
void qhasharr_free (qhasharr_t *tbl)
 qhasharr->free(): Free table reference object.

Detailed Description

Static(array) hash-table implementation.

qhasharr implements a hash table that maps keys to values and stores them in fixed-size static memory such as shared memory and memory-mapped files. The creator qhasharr() initializes static memory and divides it into small slots. The default slot size factors are defined in Q_HASHARR_NAMESIZE and Q_HASHARR_DATASIZE, and they are applied at compile time.

The value part of an element will be stored across several slots if its size exceeds the slot size. The key part of an element will be truncated if its length exceeds the limit, and an MD5 hash value will also be stored with the key. To look up a particular key, we first find an element that has the same hash value. If the key was not truncated, we simply compare the key values. If the key was truncated because its length exceeded the limit, we compare both the MD5 hash and the stored portion of the key to verify that the key is the same. Please be aware that, in theory, it is possible to pick the wrong element if a key exceeds the limit and has the same length and MD5 hash as the lookup key. In practice, however, this possibility is extremely low.

qhasharr intentionally does not provide thread-safe handling and lets users determine whether to provide a locking mechanism, depending on the use case. When race conditions are expected, you should provide shared-resource control using a mutex or semaphore to make sure data is updated by one instance at a time.

[Data Structure Diagram]
+--[Static Flat Memory Area]-----------------------------------------------+
| +-[Header]---------+ +-[Slot 0]---+ +-[Slot 1]---+ +-[Slot N]---+ |
| |Private table data| |KEY A|DATA A| |KEY B|DATA B| .... |KEY N|DATA N| |
| +------------------+ +------------+ +------------+ +------------+ |
+--------------------------------------------------------------------------+
The diagram below shows how a large value is stored.
+--[Static Flat Memory Area------------------------------------------------+
| +--------+ +-[Slot 0]---+ +-[Slot 1]---+ +-[Slot 2]---+ +-[Slot 3]-----+ |
| |TBL INFO| |KEY A|DATA A| |DATA A cont.| |KEY B|DATA B| |DATA A cont. | |
| +--------+ +------------+ +------------+ +------------+ +--------------+ |
| ^~~link~~^ ^~~~~~~~~~link~~~~~~~~~^ |
+--------------------------------------------------------------------------+
// initialize hash-table.
char memory[1000 * 10];
qhasharr_t *tbl = qhasharr(memory, sizeof(memory));
if(tbl == NULL) return;
// insert elements (key duplication is not allowed)
tbl->putstr(tbl, "e1", "a");
tbl->putstr(tbl, "e2", "b");
tbl->putstr(tbl, "e3", "c");
// debug print out
tbl->debug(tbl, stdout);
char *e2 = tbl->getstr(tbl, "e2");
if(e2 != NULL) {
printf("getstr('e2') : %s\n", e2);
free(e2);
}
// Release reference object.
tbl->free(tbl);
qhasharr_t * qhasharr(void *memory, size_t memsize)
Initialize a static hash table.
Definition qhasharr.c:208

An example for using hash table over shared memory.

[CREATOR SIDE]
int maxslots = 1000;
int memsize = qhasharr_calculate_memsize(maxslots);
// create shared memory
int shmid = qshm_init("/tmp/some_id_file", 'q', memsize, true);
if(shmid < 0) return -1; // creation failed
void *memory = qshm_get(shmid);
// initialize hash-table
qhasharr_t *tbl = qhasharr(memory, memsize);
if(hasharr == NULL) return -1;
(...your codes with your own locking mechanism...)
// Release reference object
tbl->free(tbl);
// destroy shared memory
qshm_free(shmid);
[USER SIDE]
int shmid = qshm_getid("/tmp/some_id_file", 'q');
// get shared memory
void *memory = qshm_get(shmid);
// map existing memory into table
qhasharr_t *tbl = qhasharr(memory, 0);
(...your codes with your own locking mechanism...)
// Release reference object
tbl->free(tbl);
size_t qhasharr_calculate_memsize(int max)
Get how much memory is needed for N slots.
Definition qhasharr.c:179
void * qshm_get(int shmid)
Get a pointer to shared memory.
Definition qshm.c:149
int qshm_init(const char *keyfile, int keyid, size_t size, bool recreate)
Initialize shared memory.
Definition qshm.c:91
int qshm_getid(const char *keyfile, int keyid)
Get the identifier of existing shared memory.
Definition qshm.c:127
bool qshm_free(int shmid)
Free shared memory.
Definition qshm.c:167

Definition in file qhasharr.c.

Macro Definition Documentation

◆ COLLISION_MARK

#define COLLISION_MARK   (-1)

Definition at line 150 of file qhasharr.c.

◆ EXTBLOCK_MARK

#define EXTBLOCK_MARK   (-2)

Definition at line 151 of file qhasharr.c.

Function Documentation

◆ qhasharr_calculate_memsize()

size_t qhasharr_calculate_memsize ( int max)

Get how much memory is needed for N slots.

Parameters
maxa number of maximum internal slots
Returns
memory size needed
Note
This can be used for calculating minimum memory size for N slots.

Definition at line 179 of file qhasharr.c.

◆ qhasharr()

qhasharr_t * qhasharr ( void * memory,
size_t memsize )

Initialize a static hash table.

Parameters
memorypointer to the data buffer
memsizesize of the data buffer, or 0 to use existing data
Returns
qhasharr_t pointer on success, or NULL on failure.
Return values
errnowill be set on error.
  • EINVAL : Assigned memory is too small. It must be large enough to hold at least one slot.
// Initialize a hash table with 100 slots.
// One element can use several slots.
char memory[112 * 100];
// Initialize a new table.
qhasharr_t *tbl = qhasharr(memory, sizeof(memory));
// Use an existing table.
qhasharr_t *tbl2 = qhasharr(memory, 0);

Definition at line 208 of file qhasharr.c.

◆ qhasharr_put()

bool qhasharr_put ( qhasharr_t * tbl,
const char * name,
const void * data,
size_t datasize )

qhasharr->put(): Put an object into this table.

Parameters
tblqhasharr_t container pointer.
keykey string
valuevalue object data
sizesize of value
Returns
true on success, otherwise false
Return values
errnowill be set in error condition.
  • ENOBUFS : Table doesn't have enough space to store the object.
  • EINVAL : Invalid argument.
  • EFAULT : Unexpected error. Data structure is inconsistent.

Definition at line 278 of file qhasharr.c.

◆ qhasharr_putstr()

bool qhasharr_putstr ( qhasharr_t * tbl,
const char * name,
const char * data )

qhasharr->putstr(): Put a string into this table

Parameters
tblqhasharr_t container pointer.
namekey string.
datavalue string.
Returns
true on success, otherwise false
Return values
errnowill be set in error condition.
  • ENOBUFS : Table doesn't have enough space to store the object.
  • EINVAL : Invalid argument.
  • EFAULT : Unexpected error. Data structure is inconsistent.

Definition at line 297 of file qhasharr.c.

◆ qhasharr_putstrf()

bool qhasharr_putstrf ( qhasharr_t * tbl,
const char * name,
const char * format,
... )

qhasharr->putstrf(): Put a formatted string into this table.

Parameters
tblqhasharr_t container pointer.
namekey string
formatformatted string data.
Returns
true on success, otherwise false.
Return values
errnowill be set in error condition.
  • ENOBUFS : Table doesn't have enough space to store the object.
  • ENOMEM : System memory allocation failure.
  • EINVAL : Invalid argument.
  • EFAULT : Unexpected error. Data structure is inconsistent.

Definition at line 316 of file qhasharr.c.

◆ qhasharr_put_by_obj()

bool qhasharr_put_by_obj ( qhasharr_t * tbl,
const void * name,
size_t namesize,
const void * data,
size_t datasize )

qhasharr->put_by_obj(): Put an object into this table by key object.

Parameters
tblqhasharr_t container pointer.
namekey data
namesizesize of key
datadata
datasizesize of data
Returns
true on success, otherwise false
Return values
errnowill be set in error condition.
  • ENOBUFS : Table doesn't have enough space to store the object.
  • EINVAL : Invalid argument.
  • EFAULT : Unexpected error. Data structure is inconsistent.

Definition at line 344 of file qhasharr.c.

◆ qhasharr_get()

void * qhasharr_get ( qhasharr_t * tbl,
const char * name,
size_t * datasize )

qhasharr->get(): Get an object from this table

Parameters
tblqhasharr_t container pointer.
namekey string
datasizeif not NULL, returned object size will be stored
Returns
newly allocated object on success, or NULL if not found.
Return values
errnowill be set in error condition.
  • ENOENT : No such key found.
  • EINVAL : Invalid argument.
  • ENOMEM : Memory allocation failed.
Note
The returned object must be freed after use.

Definition at line 443 of file qhasharr.c.

◆ qhasharr_getstr()

char * qhasharr_getstr ( qhasharr_t * tbl,
const char * name )

qhasharr->getstr(): Finds an object with the given name and returns it as a string.

Parameters
tblqhasharr_t container pointer.
namekey string
Returns
string pointer on success, or NULL if not found.
Return values
errnowill be set in error condition.
  • ENOENT : No such key found.
  • EINVAL : Invalid argument.
  • ENOMEM : Memory allocation failed.
Note
The returned object must be freed after use.

Definition at line 463 of file qhasharr.c.

◆ qhasharr_get_by_obj()

void * qhasharr_get_by_obj ( qhasharr_t * tbl,
const void * name,
size_t namesize,
size_t * datasize )

qhasharr->get_by_obj(): Get an object from this table by key object.

Parameters
tblqhasharr_t container pointer.
namekey data
namesizesize of key
datasizeif not NULL, returned object size will be stored
Returns
newly allocated object on success, or NULL if not found.
Return values
errnowill be set in error condition.
  • ENOENT : No such key found.
  • EINVAL : Invalid argument.
  • ENOMEM : Memory allocation failed.
Note
The returned object must be freed after use.

Definition at line 484 of file qhasharr.c.

◆ qhasharr_remove()

bool qhasharr_remove ( qhasharr_t * tbl,
const char * name )

qhasharr->remove(): Remove an object from this table.

Parameters
tblqhasharr_t container pointer.
namekey string
Returns
true on success; otherwise (if not found), returns false.
Return values
errnowill be set in error condition.
  • ENOENT : No such key found.
  • EINVAL : Invalid argument.
  • EFAULT : Unexpected error. Data structure is inconsistent.

Definition at line 516 of file qhasharr.c.

◆ qhasharr_remove_by_obj()

bool qhasharr_remove_by_obj ( qhasharr_t * tbl,
const char * name,
size_t namesize )

qhasharr->remove_by_obj(): Remove an object from this table by key object

Parameters
tblqhasharr_t container pointer.
namekey data
namesizesize of key
Returns
true on success; otherwise (if not found), returns false.
Return values
errnowill be set in error condition.
  • ENOENT : No such key found.
  • EINVAL : Invalid argument.
  • EFAULT : Unexpected error. Data structure is inconsistent.

Definition at line 533 of file qhasharr.c.

◆ qhasharr_remove_by_idx()

bool qhasharr_remove_by_idx ( qhasharr_t * tbl,
int idx )

qhasharr->remove_by_idx(): Remove an object from this table by index number.

Parameters
tblqhasharr_t container pointer.
idxslot index number
Returns
true on success; otherwise (if not found), returns false.
Return values
errnowill be set in error condition.
  • ENOENT : Index is not pointing a valid object.
  • EINVAL : Invalid argument.
  • EFAULT : Unexpected error. Data structure is inconsistent.
int idx = 0;
qhasharr_obj_t obj;
while(tbl->getnext(tbl, &obj, &idx) == true) {
if (condition_to_remove) {
idx--; // adjust index by -1
remove_by_idx(idx);
}
free(obj.name);
free(obj.data);
}
Note
This function is to remove an object in getnext() traversal loop without knowing the object name but index value. When key names are longer than defined size, the keys are stored with truncation with their fingerprint, so this method provides a way to remove those keys. getnext() returns actual index + 1(pointing next slot of current finding), so you need to adjust it by -1 for a valid index. After you remove an object with this method, rewind idx by -1 before calling getnext() because collision objects can be moved back to removed index again, so by adjusting index by -1, getnext() can continue search from the removed slot index again. Please see the example above.

Definition at line 589 of file qhasharr.c.

◆ qhasharr_getnext()

bool qhasharr_getnext ( qhasharr_t * tbl,
qhasharr_obj_t * obj,
int * idx )

qhasharr->getnext(): Get next element.

Parameters
tblqhasharr_t container pointer.
idxindex pointer
Returns
key name string if successful, otherwise(end of table) returns NULL
Return values
errnowill be set in error condition.
  • ENOENT : No next element.
  • EINVAL : Invalid argument.
  • ENOMEM : Memory allocation failed.
int idx = 0;
qhasharr_obj_t obj;
while(tbl->getnext(tbl, &obj, &idx) == true) {
printf("NAMESIZE=%zu, DATASIZE=%zu\n",
obj.namesize, obj.datasize);
free(obj.name); // key name
free(obj.data); // value data
}
Note
Please be aware a key name will be returned with truncated length because key name gets truncated if it doesn't fit into slot size, Q_HASHARR_NAMESIZE.

Definition at line 674 of file qhasharr.c.

◆ qhasharr_size()

int qhasharr_size ( qhasharr_t * tbl,
int * maxslots,
int * usedslots )

qhasharr->size(): Returns the number of objects in this table.

Parameters
tblqhasharr_t container pointer.
maxslotsif not NULL, total number of slots will be stored.
usedslotsif not NULL, total number of used slots will be stored.
Returns
a number of elements stored.

Definition at line 725 of file qhasharr.c.

◆ qhasharr_clear()

void qhasharr_clear ( qhasharr_t * tbl)

qhasharr->clear(): Clears this table so that it contains no keys.

Parameters
tblqhasharr_t container pointer.
Returns
true on success, otherwise false.

Definition at line 748 of file qhasharr.c.

◆ qhasharr_debug()

bool qhasharr_debug ( qhasharr_t * tbl,
FILE * out )

qhasharr->debug(): Print the hash table for debugging purposes.

Parameters
tblqhasharr_t container pointer.
outoutput stream such as stdout or stderr.
Returns
true on success, otherwise false
Return values
errnowill be set in error condition.
  • EIO : Invalid output stream.

Definition at line 778 of file qhasharr.c.

◆ qhasharr_free()

void qhasharr_free ( qhasharr_t * tbl)

qhasharr->free(): Free table reference object.

Parameters
tblqhashtbl_t container pointer.
Note
This does not free the data memory but only the memory of qhasharr struct. User provided data memory must be freed by user.

Definition at line 860 of file qhasharr.c.