Various Big Data Databases

  • CockrockDB:
    • An open source version of Google Spanner.
    • A distributed SQL database built on top of a transactional and consistent key:value store.

 

Advertisements
Various Big Data Databases

Java Thread Pool (ExecutorService)

Java concurrent package ExecutorService interface provides thread pool support. Different types of thread pools (ExecutorService) can be created using Executors class’s following static functions

  • newCachedThreadPool()
    • Create a new thread for a new submitted task (callable or future). Idle thread will be terminated after 60 seconds.
  • newFixedThreadPool(int nThreads)
    • Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue
  • newSingleThreadExecutor()
    • Creates an Executor that uses a single worker thread operating off an unbounded queue
  • newScheduledThreadPool(int corePoolSize)
    • Creates a thread pool that can schedule commands to run after a given delay, or to execute periodically
  • newWorkStealPool(int parellelism)
    • Creates a thread pool that maintains enough threads to support the given parallelism level, and may use multiple queues to reduce contention.
    • Note that, this means each thread may have its own task queue, and other thread can “steal” task from neighbor thread if needed.
    • This function actual create ForkJoinPool, i.e.,
public static ExecutorService newWorkStealingPool() {
    return new ForkJoinPool(Runtime.getRuntime().availableProcessors(),
                            ForkJoinPool.defaultForkJoinWorkerThreadFactory,
                            null, true);
}
Java Thread Pool (ExecutorService)

C/C++

Built-in Function: long __builtin_expect (long exp, long c) indicates compiler to do optimization. It return value is exp, where c is actually the indicator to compiler which value exp most likely will take. For example,

if (__buildin_expect(x,0)) {

doSomthing();

}

would indicate that we do not expect to call foo, since we expect x to be zero, i.e., it is equivalent to 

if (x!=0) {

doSomething();

}

For following code:

if (__builtin_expect(x, 0)) {
    foo();
    ...
} else {
    bar();
    ...
}

Compiler could generate following code

cmp   $x, 0
  jne   _foo
_bar:
  call  bar
  ...
  jmp   after_if
_foo:
  call  foo
  ...
after_if:


In this case,  since programmer expect x equal to 0

C/C++

DevOP

Ansible – https://www.ansible.com

  • A IT framework to run tasks in multiple remote machine
  • Use yaml to describe task in Ansible Playbook
  • Ansible galaxy to host/share existing ansible role between people. Ansible role is a standalone self-contain ansible
  • Ansible is agentless, It is based on ssh. No need to install agent service in remote to-be-managed machine.
  • Concept:
    • Inventory – the machine list to be managed
    • Playbood – Yaml files to describe the operation/task

 

DevOP