Brainstem Breakfast

multi_array and multi_vector Template Aliases With SFINAE

Wed, 15 Apr 2015 22:02:44 +0000

Let’s face it, the std::array and std::vector containers suck for multi-dimensional data. The situation is so bad it makes me long for the days of T array[N][M]; and T** array. Well, it turns out, to fix our modern C++11 and C++14 array containers, doesn’t take much work.

The Problem

Defining a multi-dimensional array using C++11/14 containers is no good:

The good old days (which can still be used) were much better in some ways:

And worse in other ways:

The Solution

Why not a syntax for multi-dimensional container style arrays and vectors? You could get the best of both worlds like this:

The Code

It turns out, with the help of alias templates, and some class based SFINAE not much code is needed to achieve our desired syntax.

multi::array:

multi::vector:

We don’t need to reinvent the same data structures over and over when we can just use what we already have and let the templates do all the work.

Repo for this code: https://github.com/brianrackle/multi

Output Parameters as Return Values

Thu, 29 Jan 2015 21:36:44 +0000

Recently, I got tired of dealing with output parameters. I am not trying to convince anyone not to use them, but at times it is just nice to have values returned directly from the function. So let’s travel down this road of terrible ideas and loathesome mistakes.

The Problem

As seen below, output parameters can be cumbersome.

Point p0, p1;

double x0, x1;
p0.get_x(x0);
p1.get_x(x1);

double midpoint;
get_midpoint(x0, x1, midpoint);

The above code can be greatly simplified using return values, as seen here:

Point p0, p1;

double midpoint = get_midpoint(p0.get_x(), p1.get_x());

So I set out on a mission, I must defeat the output parameter and make it submit to my will. My goal was simple, create a way to turn output parameters into return values.

The Code

As usual, the full code can be found here

Here I will go through the evolution of the code with a brief explanation of each step. I have found that developing templates in steps by decomposing non-template code results in much a much smoother development process, so this post will reflect that. In all, it is a bad idea to write template code for an extended period of time without compiling, it will lead to a bad time.

Attempt One

The first pass at the problem was simple and produced a clumsy and inflexible solution. Consisting of two template functions and a macro:

//O object type, F function pointer type, P output parameter type
template<class O, class F, class P>
P to_out(O & obj, F func)
{
  P out;
  (obj.*func)(out);
  return std::move(out);
};

#define t_o(O, F, P) to_out<decltype(O), decltype(F), P>(O, F)

The usage of this code is as follows:

t_o(obj,&decltype(obj)::get_x0, double);

Let’s go through the problems with this implementation.

Template type deduction is not possible because the P template parameter is explicit.
A macro is needed to hide the specification of the template parameters.
Support for only functions with a single parameters.

Attempt Two

To fix the problems with the previous attempt, it is necessary to allow for template type deduction of the method’s output parameter. To accomplish this, we need a traits class that works on methods. The basic ideas for method traits are covered here. The basic implementation that works with methods that have a single parameter is as follows:

template<class T> struct member_function_traits;
template<class T, class R, class A>
struct member_function_traits<R (T::*)(A)>
{
  using class_type = T;
  using return_type = R;
  using arg_type = A;
};

Types T, R, and A are deduced through the member_function_traits type specialization <R (T::*)(A)>. These types are then exposed publicly with type aliasing through class_type, return_type and arg_type.

Now it is possible to remove the ugly macro and rely on template type deduction to give us the same information that was previously explicitly defined. Doing something like this:

//O object type, F function pointer type
template<class O, class F>
P to_out(O && obj, F func)
{
  std::remove_reference<member_function_traits<F>::arg_type>>::type out;
  (obj.*func)(out);
  return std::move(out);
};

This implementation saves us several problems. First, thanks to universal references we only need to define a single to_out function that can handle l-value and r-value references. And universal references are available because template type deduction is being used. And template type deduction is being used because the method’s output parameter type is no longer defined through the template, but through examining the F func parameter using our member_function_traits class.

Attempt 3

The final piece to the puzzle is allowing functions with variable numbers of parameters. In some cases, functions with output parameters will require input. For example, get_name(int ssn, std::string & name); So member_function_traits needs to be elaborated on to allow the deduction of all parameters of a method, no matter how numerous.

template<class T> struct mft;
template<class T, class R, class... A>
struct mft<R (T::*)(A...)>
{
  using class_type = T;
  using return_type = R;

  static constexpr std::size_t arity = sizeof...(A);

  template <std::size_t N>
  struct arg
  {
    using type = typename std::tuple_element<N, std::tuple<A...>>::type;
  };
};

Here, a few techniques come into play. First, variadic templates allow our member function traits class mftto accept a function with any number of arguments. Additionally, A has been replaced by a variadic, arity has been added to keep a count of the number of parameters the method takes, and arg is now to be used in place of the type alias arg_type. And arg parameter types are looked up using the N template parameter.

The individual parameter type is looked up by constructing a tuple type using parameter pack expansion and then looking up the type of the Nth tuple element with the handy std::tuple_element.

Of lesser importance but interesting to note, is that mft contains most of the varying uses of the ... syntax. Nothing like making a syntax highly contextual in order to improve language simplicity. I will expand on the topic of variadic templates in another blog post.

With mft complete, the to_out function can be improved to take functions with a variable number of arguments:

template <class T, std::size_t N> using mft_arg_t = 
  typename mft<T>::template arg<N>::type;

template <class T> using rr = typename std::remove_reference<T>;

template<class O, class F, class... A >
auto to_out(O && obj, F func, A... args)
{
  constexpr auto arity = mft<F>::arity;

  typename std::remove_reference<mft_arg_t<F, arity - 1>>::type out_param;

  (obj.*func)(args..., out_param);
  return std::move(out_param);
};

to_out is a template function that uses that amazing template type deduction that we want so dearly. The final function parameter, A... args, defines a function parameter pack, and as noted earl before, class... A, defines the template parameter pack. However, there is one gotcha in this code on this line:

  template <class T, std::size_t N> using mft_arg_t = 
    typename mft<T>::template arg<N>::type;

You might noticed the keyword template appears twice. Having a look at ISO C++03 14.2/4 might clear things up, or make things more confusing:

When the name of a member template specialization appears after . or -> in a postfix-expression, or after nested-name-specifier in a qualified-id, and the postfix-expression or qualified-id explicitly depends on a template-parameter (14.6.2), the member template name must be prefixed by the keyword template. Otherwise the name is assumed to name a non-template.

In this case, because the template argument to arg is part of a “nested-name-specifier” and relies on the template parameter N, we need to use the template keyword.

Conclusion

So that wasn’t so much work but did we get anything useful out of that project? I am not so sure, but I am leaning towards no. The syntax for converting output parameters to return values just plain sucks.

get_midpoint(to_out(obj, &Type::get_x0), to_out(obj, &Type::get_x1));

Nesting calls results in an extremely difficult to read mess. Getting single values is a little more understandable and maybe with a better name than to_out it might even be understandable, but it is still an excessive amount of code. One problem is that the function argument needs to be defined in terms of the argument for the object. I tried various trickery but couldn’t figure out a way, when dealing with methods, to make it work without &Type::. In all, an interesting experiment, however the results leave me wanting more.

Ultimately, this project leads me to question the plausibility and usefulness of modifying functions through templates. Of course the practice of wrapping functions is common and useful. In hindsight, I think the more useful wrapper would be to convert return values to output parameters for purposes of unifying various API’s to one style. Also, while to_out remains incomplete, there would be quite a bit more work to make it work with *, ** and however else people are defining outputs. Whereas, return value to output parameter would be much more straight forward.

The Iterator Iterator

Sun, 19 Oct 2014 22:02:44 +0000

So I was recently thinking about the code of an iterating for loop and why there is no functionality to retrieve an iterator with a range-based for loop. The range-based for loop was a hugely popular addition to C++, however it is limited to only returning dereferenced iterators. I don’t think I need to convince anyone of why one would need an iterator rather than a dereferenced iterator, there are plenty of use cases to warrant such a loop. The issue here is that to iterate over a range using a classic for loop takes quite a lot of characters, even with the wonderful auto it that C++11 provides.

The Problem

Range-based for loops work like this:

//get a copy of each value in the container
for(auto value : container) { }

//get a reference of each value in the container
for(auto & v : container) { }

But sometimes you need this:

//get an iterator for each value in the container
for(auto it = container.begin(); it != container.end(); ++it) { }

Ideally, it would work like this:

//ideal syntax for getting an iterator for each value in the container
for_it(auto it : container) { }

I haven’t mastered meta-programming, so adding a for_it definition is still on the horizon. So I propose this:

//syntax I have settled for to get an iterator for each value in the container
for(auto it : iit(container)) { }

The Code

The full solution can be found here

The Iterator

The code here is small but powerful. itr (the iterator iterator) is simply a container that holds a standard forward iterator. All operations work the same as most other forward iterators except that the iterator iterator dereference operator returns an iterator, not a dereferenced iterator.

template <typename T>
class itr : public T
{
  T _obj;
 public:
    itr(T&& obj) : _obj(std::move(obj)) {}
    bool operator!=(const itr& other) const { return _obj != other._obj; }
    T operator*() const { return _obj; }
    const itr& operator++() { ++_obj; return *this; }
};

This is a minimal implementation of only what is needed for range-based for loops. A proper implementation would implement std::iterator and also follow the rules of `InputIterator’

In addition to the iterator class, a make function has been implemented. The idea here is to save the user from having to define the type of itr themselves, a similar idea to make_tuple and make_shared. My make function has been simply defined as iit to save some characters.

template <typename T>
auto iit(T&& t) -> typename std::conditional<std::is_lvalue_reference<decltype(t)>::value,
                                 ref<T>,
                                 val<T>>::type
{
    return std::forward<T>(t);
}

Ref and Val

iit is defined around a universal reference (l-value or r-value) and a return type conditionally tied to if the type is l-value or r-value. This allows iit to accept the value by reference using the class ref, or move construct the type val. Primarily, the ref and val encapsulating containers provide the begin and end functions which return itr. Additionally, having a ref and val class allows iit to encapsulate the passed in container and keep an r-value alive through the lifetime of ref. Here are the definitions of ref and val:

template <typename T>
class val
{
  T _obj;
 public:
 val(T&& obj) : _obj(std::move(obj)) {}

  auto begin() const -> itr<decltype(std::begin(this->_obj))> { return std::begin(_obj); }
  auto end() const -> itr<decltype(std::end(this->_obj))> { return std::end(_obj); }
};

template <typename T>
class ref
{
  T& _obj;
public:
 ref(T& obj) : _obj(obj) {}
  
  auto begin() const -> itr<decltype(std::begin(this->_obj))> { return std::begin(_obj); }
  auto end() const -> itr<decltype(std::end(this->_obj))> { return std::end(_obj); }
};

Usage

for(auto it : iit(vector<int>{1,2,3,4})) 
  std::cout << *it << std::endl;

Without the val container to capture the r-value and store it, the vector would go out of scope.

vector<int> v = {1,2,3,4};
for(auto it : iit(v)) 
  if(it == v.begin())
    std::cout << *it << std::endl;

Here, v is sent to ref which only holds a reference to vector, no copies needed.

Thanks for reading!

Brian Rackle

Extending Regex Replace

Sat, 20 Sep 2014 16:27:00 +0000

Recently I was working with the STL regular expression library, found in <regex>. I was building a simple utility to identify tokens in a string and replace them with pre-defined values. First, I needed all replacements to occur in a single pass. Second, I needed to identify each token with it’s paired replacement value. As an example, let’s say I am using tokens to define specific directory paths. These are the tokens and their matching values:

Token	Value
{MyApp}	brainstem_breakfast
{InstallDir}	/home/brian/apps/

I want to do a regex_replace on the string: {InstallDir}{MyApp}

regex_replace is unable to to perform this job because it only takes a string as a formatter. The fmt variable can use character groups using the $xx syntax, however it can’t differentiate between groups dynamically. I am no regular expression wizard, but I think regex_replace will not be able to fulfill my needs here.

Regex Iterators

Luckily, I can construct a new regex_replace alternative using regex_token_iterator, a ForwardIterator that allows tokenizing both matched and unmatched strings. With regex_token_iterator it is possible to iterate unmatched expressions, matched expressions, and submatched expressions. Controlling what the tokenizer iterates is done through the int submatchor array submatches parameter of the constructor. -1 iterates unmatched portions, 0 matches the entire regular expression, and values 1 and greater match groups/sub-matches.

Extending regex_replace

Using the regex_token_iterator it will be possible to construct a new regex_replace function. The new function, rather than taking an std::basic_string or CharT * , will take a std::function<std::basic_string (int submatch, std::basic_string)>. In fact, the Boost regex_replace, upon which the STL version was designed, provides a similar functionality. In Boost fmt is defined as a Formatter, which can be C style string, a container of CharT (like std::basic_string, although I suppose std::vector<CharT> would also work), or a “unary, binary or ternary functor that computes the replacement string from a function call”.

The Solution

The Full Solution

The basic idea is to iterate unmatched and submatched expressions. With unmatched expressions, the token will be passed along to the output string. With matched expressions, the token and the subexpression index will be passed to the fmt function. The fmt function will determine the replacement value and return it to be inserted into the return string.

Here I have created a string to show off the capabilities of the new regex_replace_ext as well as a regular expression to match the tokens in the string.

const std::string bss("{Id_1} [Fill_0] {Id_2} [Fill_1] {Id_3} {Id_4} {Id_5}.");
const std::regex re("(\\{.*?\\})|(\\[.*?\\])");

Additionally, I need a mapping between tokens and values. I have seperated the “{ }” and “[ ]” tokens to highlight the flexibility of the new regex_replace_ext. I also, have created the function arguement for the fmt parameter. Match the submatch index to the correct dictionary, and then look for the token in the dictionary.

    using dictionary = std::map<std::string,std::string>;
    const std::vector<const dictionary> dict
      {
	{
	  {"{Id_1}","This"},
	  {"{Id_2}","test"},
	  {"{Id_3}","my"},
	  {"{Id_4}","favorite"},
	  {"{Id_5}","hotdog"}
	},
	{
	  {"[Fill_0]","is a"},
	  {"[Fill_1]","of"}
	}
      };

    auto fmt1 = [&dict](const unsigned smatch, const std::string & s)->std::string
      {
	auto dict_smatch = smatch - 1;
	if(dict_smatch > dict.size()-1)
	  return s; //more submatches than expected
  
        const auto it = dict[dict_smatch].find(s); 
	return it != dict[dict_smatch].cend() ? it->second : s;
      };

With all the arguments ready it is time to put the new regex_replace_ext into action.

  template< class Traits, class CharT,
	    class STraits, class SAlloc >
  inline std::basic_string<CharT,STraits,SAlloc> 
  regex_replace_ext( const std::basic_string<CharT,STraits,SAlloc>& s,
		     const std::basic_regex<CharT,Traits>& re,
		     const typename std::common_type<std::function<std::basic_string<CharT,STraits,SAlloc> 
		     (const unsigned, const std::basic_string<CharT,STraits,SAlloc> &)>>::type& fmt,
			  std::regex_constants::match_flag_type flags =
			  std::regex_constants::match_default)
  {
    std::vector<int> smatches{-1};
    if(re.mark_count() == 0)
	smatches.push_back(0);
    else
      {
	smatches.resize(1+re.mark_count());
	std::iota(std::next(smatches.begin()), smatches.end(), 1); //-1, 1, 2, etc...    
      }

    unsigned smatch_count = smatches.size();
    unsigned count = 0;

    std::regex_token_iterator
      <typename std::basic_string<CharT,STraits,SAlloc>::const_iterator> 
      tbegin(s.begin(), s.end(), re, smatches, flags), tend;            

    std::basic_stringstream<CharT,STraits,SAlloc> ret_val;
    std::for_each(tbegin, tend, [&count,&smatch_count,&ret_val,&fmt]
		  (const std::basic_string<CharT,STraits,SAlloc> & token)
		  {
		    if(token.size() != 0)
		      {
			if(!count) 
			  ret_val << token;
			else
			  ret_val << fmt(count,token);
		      }
		    count = ++count % smatch_count;
		  });
    return ret_val.str();

  }

Let’s break down the functionality into easy to understand parts. First, regex_token_iterator takes a submatch value or a vector of values, and I want to include the unmatched parts of the string. So I populate, -1 (the unmatched), skip 0 (the whole expression), and iterate incrementing values for as many submatches as exist in the regular expression.

    std::vector<int> smatches{-1};
    if(re.mark_count() == 0)
	smatches.push_back(0);
    else
      {
	smatches.resize(1+re.mark_count());
	std::iota(std::next(smatches.begin()), smatches.end(), 1); //-1, 1, 2, etc...    
      }

    unsigned smatch_count = smatches.size();
    unsigned count = 0;

Constructing the regular expression iterator.

    std::regex_token_iterator
      <typename std::basic_string<CharT,STraits,SAlloc>::const_iterator> 
      tbegin(s.begin(), s.end(), re, smatches, flags), tend;

Finally, allowing the regex_token_iterator to do most of the work, I simply iterate the tokens. The regex_token_iterator will iterate the submatch tokens in the other they were passed, so if a token isn’t found the value for the token will be an empty string. It necessary to keep track of which submatch the iterator is currently on, this is possible simply by incrementing a counter with a modulus of the total submatches. Finally, when the token is on count == 0 that means the iterator is matching the unmatched token, so we simply put that value in the return string. Otherwise, count and token are sent to the fmt function which we defined earlier for the replacement values.

    std::basic_stringstream<CharT,STraits,SAlloc> ret_val;
    std::for_each(tbegin, tend, [&count,&smatch_count,&ret_val,&fmt]
		  (const std::basic_string<CharT,STraits,SAlloc> & token)
		  {
		    if(token.size() != 0)
		      {
			if(count == 0) 
			  ret_val << token;
			else
			  ret_val << fmt(count,token);
		      }
		    count = ++count % smatch_count;
		  });
    return ret_val.str();

Thanks for reading! Brian Rackle

Ubuntu on Azure

Mon, 08 Sep 2014 09:25:30 +0000

I have decided to move my projects to use Clang on Linux. The primary reason that spurred this decision is to be able to keep pace with the latest C++ features. C++14 refines C++11 and makes the language much more intuitive and concise. This being said, I need a compiler that supports the latest language features and currently Clang is king of feature support. I could whine about Visual Studio but it is a great IDE. I use Visual Studio daily, and will continue to use it in projects when I see fit. However, right now for this blog, Ubuntu, Emacs and Clang is my new platform.

It has been six years since I last developed on Linux, and it seems some things have changed since then. As such, I have put together a guide to make setup as easy as possible. This guide is exactly how I have my machine setup and once you have your Ubuntu machine running all you have to do is run a single line of shell commands I have put together.

The Machine

First, thing you are going to need is your Linux machine. Sure you could grab an old laptop, dual boot, or install a local VM, hell you might even have a server at home to run your VM on. If so, you will want to head over to http://www.ubuntu.com/download/server and download an ISO and install it. This is not what I did. To make things easy, and to have a VM that I could access from anywhere, I created a cloud VM on Microsoft’s Azure platform.

If you are unfamiliar with Azure, go to http://azure.microsoft.com/ and click the Free Trial button in the top right corner. You will get free credits to spend on Azure services and if you manage them correctly they will last you quite a long time.

Azure Ubuntu Server 14.04 LTS

Once you have your Azure account setup, you need to create your Linux VM. Go to your Portal. Click the Virtual Machines tab and down at the bottom left click “New”, which looks something like:

Then click “From Gallery”:

Then Choose an Image, selecting Ubuntu, and then Ubuntu Server 14.04 LTS:

Now configure your VM with the following options. Basic A1 is the second cheapest at $55 per month and Basic A0 is cheaper at $13 per month. I recommend using Basic A1 at least while you setup the machine so that it doesn’t take hours, you can switch to Basic A0 afterwards. The user name and password will be used to log into your VM:

Next you will choose the region for your VM, generally you want the VM close to you. Also set the “Cloud Server DNS Name”, which is the the address you will use to SSH into the VM. To add other “services” to your VM such as RDP or VNC you will add the appropriate endpoints to the Cloud Service you are setting up now.

Click through the rest of the setup and wait about 5 minutes for Azure to provision your brand spanking new Ubuntu VM. Head over to your Virtual Machines portal and when Status says “Running”, click the circled area:

You will be at your VM’s Azure Portal and can get all sorts of useful information like the server load and the address for the service and SSH endpoint you set up earlier (under quick glance “SSH Details”). Copy the address under SSH Details.

Accessing the VM

To access your VM you need an SSH client. On Windows, I recommend KiTTY, it is a branch of the long-time popular open source PuTTY. Download KiTTY

Open KiTTY. Fill out the SSH Details you copied earlier. Give the connection a nickname. Save.

Now click Open in the bottom left of the KiTTY window. This will connect your with your Azure VM. You will get a warning because you didn’t upload an SSH certificate, click Ok. And then enter your username and password which you setup earlier in the Azure portal.

You now have a Linux VM in the cloud, and that probably didn’t take you any more than 10 minutes!

Setting Up Your Ubuntu VM

On Github

I have to credit Solarian Programmer for providing many of the next steps. However, I have modified and added to them so that they will work with your Azure Ubuntu 14.04 VM. The steps on Solarian will not work for Ubuntu 14.04.

You can follow each step individually, or you can just copy the all in one version

All Steps in One

This is all the steps for setting up your Ubuntu VM concatenated into a single command. After pasting this into your command prompt, you will have to stick around for a few minutes to type ‘y’ and ‘enter’ a few times. Do not close KiTTY or let your computer go to sleep because your session will time-out and your VM will not be set up properly.

sudo apt-get update -y && sudo apt-get upgrade -y && sudo apt-get install g++ subversion cmake emacs git xdg-utils htop ncurses-dev -y && sudo mkdir repos && cd ~/repos/ && sudo mkdir clang && cd clang && sudo svn co http://llvm.org/svn/llvm-project/llvm/branches/release_35/ llvm && cd llvm/tools && sudo svn co http://llvm.org/svn/llvm-project/cfe/branches/release_35/ clang && cd ~/repos/clang && sudo mkdir build && cd build && sudo ../llvm/configure --enable-optimized --enable-targets=host --disable-compiler-version-checks && sudo make -j 8 && sudo make install && cd ~/repos/clang && sudo svn co http://llvm.org/svn/llvm-project/libcxx/trunk libcxx && sudo mkdir build_libcxx && cd build_libcxx && sudo CC=clang CXX=clang++ cmake -G "Unix Makefiles" -DLIBCXX_CXX_ABI=libsupc++ -DLIBCXX_LIBSUPCXX_INCLUDE_PATHS="/usr/include/c++/4.8;/usr/include/x86_64-linux-gnu/c++/4.8" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr $HOME/repos/clang/libcxx && sudo make -j 8 && sudo make install && xdg-mime default emacs.desktop text/plain && cd ~/repos/ && sudo git clone https://github.com/brianrackle/brainstem_breakfast.git && cd brainstem_breakfast/source && sudo clang++ --std=c++14 --stdlib=libc++ main.cpp

Each Step Explained

Here is all the steps you need to get your Ubuntu installation up and running with a great compiler. Just copy each line individually into the command line, and by the end you will be redy to develop on Linux. These commands are the same as the above commands, but are seperated for easy viewing.

Initialize Environment

Here we update the software on the VM and install some new software.

sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install g++ subversion cmake emacs git xdg-utils htop ncurses-dev -y
sudo mkdir repos

Pull Clang and LLVM

Get the source code for Clang and LLVM. LLVM is the backend technology that Clang is built upon.

cd ~/repos/
sudo mkdir clang
cd clang
sudo svn co http://llvm.org/svn/llvm-project/llvm/branches/release_35/ llvm
cd llvm/tools
sudo svn co http://llvm.org/svn/llvm-project/cfe/branches/release_35/ clang

Build Clang and LLVM

Build and install Clang.

cd ~/repos/clang
sudo mkdir build
cd build
sudo ../llvm/configure --enable-optimized --enable-targets=host --disable-compiler-version-checks
sudo make -j 8
sudo make install

Pull and Make libc++

Get the libc++ source and build it with Clang. libc++ is a newer C++ STL implementation.

cd ~/repos/clang
sudo svn co http://llvm.org/svn/llvm-project/libcxx/trunk libcxx
sudo mkdir build_libcxx
cd build_libcxx
sudo CC=clang CXX=clang++ cmake -G "Unix Makefiles" -DLIBCXX_CXX_ABI=libsupc++ -DLIBCXX_LIBSUPCXX_INCLUDE_PATHS="/usr/include/c++/4.8;/usr/include/x86_64-linux-gnu/c++/4.8" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr $HOME/repos/clang/libcxx
sudo make -j 8
sudo make install

Setup Emacs

Emacs is my code editor of choice on Linux. This will set Emacs as the default text editor.

xdg-mime default emacs.desktop text/plain

Use Git and Compile BSB

Use Git to pull some source code (the brainstem breakfast code) and compile it using clang.

cd ~/repos/
sudo git clone https://github.com/brianrackle/brainstem_breakfast.git
cd brainstem_breakfast/source
sudo clang++ --std=c++14 --stdlib=libc++ main.cpp

If you are able to compile the BSB source then you have successfully set up you Clang on Ubuntu and you can now write C++14 code.

Enjoy your new Linux environment. If you have any problems, let me know in the comments or over twitter and I will do my best to help you.

Cheers! Brian

Generating a Color Gradient Part 1

Sat, 23 Aug 2014 14:36:00 +0000

The purpose of data visualization is to put information into a format that is easy to comprehend by the human brain. One way to visualize data is through color. Color is a very effective way to convey changes in data. While a random color palette is good for representing categorical data. Continuous data, such as temperature, time, or population is better represented with a continuous color palette, such as a gradient.

Color Gradients

A color gradient is one very useful tool for representing changes in a continuous dataset. A properly chosen gradient will allow a viewer to quickly assess both small and large changes in data. Given that gradients are such a useful tool, it is handy to know how to generate a gradient.

A gradient is simply a range of colors determined by their position. For the purpose of this post I will just be covering one dimensional gradients, also known as axial gradients. Being one dimensional, the axial gradient is a simple linear interpolation of values between two points. Commonly, only two colors are interpolated between in axial gradient. Multiple color gradients also exist. For example, the rainbow gradient displays the entire rgb color range (given equal brightness and saturation).

The Math

Calculating a two color gradient is very easy. For two color axial gradients, it is necessary to project a percent to a range determined by each of the components of an RGB color. For example, to build a gradient from the color rgb(42, 163, 90) to color rgb(207, 74, 33) there will be three different interpolations. The red component will be interpolated from 42 to 207. The green component will be interpolated from 163 to 74. The blue component will be interpolated from 90 to 33.

My previous post, “Range Mapping and Projecting Values”, covers all the math needed to interpolate between ranges. To recap, given an input value from 0 to 1 and an output range from red component 42 to red component 207 (1 byte color) the following equation can be used to get the projected value:

\[projected = \frac{(value - range1from) \times (range2to - range2from)}{range1to - range1from} + range2from\]

Given that our input range is clamped to the range 0 to 1, it is possible to simplify the equation to:

\[projected = (value \times (range2to - range2from)) + range2from\] \[projected = (value \times (207 - 42)) + 42\] \[projected = value \times 165 + 42\]

The Code

The Full Solution

To leverage the code developed for the, “Range Mapping and Projecting Values” post, it is convenient to simply use the scale_value function to interpolate each color component independently. This allows the gradient_value function to be reduced to just a few lines, with each RGB color component being mapped from a range of 0 to 1 to the range of from_color.(r/g/b) to to_color.(r/g/b)

#include "range_map.hpp"

class rgb
{
public:
	using value_type = uint8_t;
	value_type r, g, b;
};

inline rgb gradient_value(const double_t value, const rgb & from_color, const rgb & to_color)
{
	using namespace range_map;
	return {
		scale_value(value, 0.0, 1.0, from_color.r, to_color.r),
		scale_value(value, 0.0, 1.0, from_color.g, to_color.g),
		scale_value(value, 0.0, 1.0, from_color.b, to_color.b) };
}

Optimizations

It it would be handy to leverage the equation simplification from above to reduce unnecessary processing and reduce the number of parameters needed when using using a clamped percentage. So the following code was developed and added to the range_map.hpp header:

//scales 'value' from the range 0.0/1.0 to the range 'lowestTo'/'highestTo'
template <class _TT,
class = typename std::enable_if<std::is_arithmetic<_TT>::value>::type>
	_TT scale_value(const double_t value, const _TT lowestTo, const _TT highestTo)
{
	if (value <= 0.0)
		return lowestTo;
	if (value >= 1.0)
		return highestTo;

	//scale by half to account for negative and positive range being too large to represent
	const auto && tHLF = [](_TT v){ return v / long double(2.0); };

	auto scaledOffsetResult = (tHLF(highestTo) - tHLF(lowestTo)) * value;
	return (_TT)(scaledOffsetResult + lowestTo + scaledOffsetResult); //seperated to prevent overflow
}

Now the gradient code can be simplified:

#include "markdown.hpp"
#include "range_map.hpp"

class rgb
{
public:
	using value_type = uint8_t;
	value_type r, g, b;
};

inline rgb gradient_value(const double_t value, const rgb & from_color, const rgb & to_color)
{
	using namespace range_map;
	return {
		scale_value(value, from_color.r, to_color.r),
		scale_value(value, from_color.g, to_color.g),
		scale_value(value, from_color.b, to_color.b) };
}

int main()
{
	using namespace markdown;

	ostream << table_header("cccc", "Transition1", "Transition2", "Transition3", "Transition4");
	for (int i = 0; i <= 10; ++i)
	{
		double_t v = (double_t)i / 10.0;
		auto make_cell = [](std::string clr)->std::string
		{return span(clr, color + "#ffffff", bg_color + clr); };

		ostream << table_row(
			make_cell(to_hex(gradient_value(v, { 255, 0, 0 }, { 0, 0, 0 }))),
			make_cell(to_hex(gradient_value(v, { 255, 255, 255 }, { 0, 0, 0 }))),
			make_cell(to_hex(gradient_value(v, { 0, 0, 0 }, { 255, 255, 255 }))),
			make_cell(to_hex(gradient_value(v, { 42, 163, 90 }, { 207, 74, 33 }))));
	}
}

The Output

Transition1	Transition2	Transition3	Transition4
#ff0000	#ffffff	#000000	#2aa35a
#e50000	#e5e5e5	#191919	#3a9a54
#cc0000	#cccccc	#323232	#4a914e
#b20000	#b2b2b2	#4c4c4c	#5b8848
#990000	#999999	#656565	#6b7f42
#800000	#808080	#7f7f7f	#7c773d
#660000	#666666	#989898	#8c6e37
#4d0000	#4d4d4d	#b1b1b1	#9c6531
#330000	#333333	#cbcbcb	#ad5c2b
#1a0000	#1a1a1a	#e4e4e4	#bd5325
#000000	#000000	#ffffff	#cf4a21

There are no tricks or gotcha’s with this code, all the work is performed in the scale_value function. In part 2 on gradients, I will cover another axial gradient and also create some fun test cases for the new functionality.

Range Mapping and Projecting Values

Mon, 18 Aug 2014 22:10:00 +0000

Mapping values from one range to another is a common need in certain problem sets. Simply getting a percent of a total is a basic form of range mapping. You have values in a given range, and you need to translate those values into a range from 0 to 1. Let’s see if we can generalize the operation to map any range to any other range.

The Basics

To start, it is important to understand the easiest form of range mapping, finding a percent. Finding a percent in a range starting from zero is actually the easiest starting point. This is easy because both range_to and value represent their distance from range_from. Meaning, if we have a range from 0 to 255 then 255 is range_to, and it is also the distance from range_from. Distance from the origin is important, because it allows us to deduce the total percentage within the range.

Given a value of 32, and a range of 0 to 255. The following equations will give 32 as a percent of 255:

\[percent = \frac{value}{range1}\] \[percent = \frac{32}{255}\] \[percent = 0.1254901960\]

It is now possible to use percent to project into a new range. Given a second range, 0 to 511 use the following equations to project the previously calculated percent into that range:

\[projected = percent \times range2\] \[projected = 0.1254901960 \times 511\] \[projected = 64.125490156\]

The previous equations can be combined into the following equation:

\[projected = \frac{value}{range1} \times range2\]

Generalizing

The math is very easy. All that is needed is to generalize it to work with any pair of ranges. First, it is necessary to allow ranges that start at any number, not just 0. This is done by first shifting the range over to start at 0 and then shifting back after projecting using the following modification to our previous equation:

\[projected = \frac{value - range1from}{range1to - range1from} \times (range2to - range2from) + range2from\] \[projected = \frac{(value - range1from) \times (range2to - range2from)}{range1to - range1from} + range2from\] \[projected = \frac{(32 - 0) \times (511 - 0)}{255 - 0} + 0\] \[projected = 64.125490196\]

The Code

The Full Solution

Using the above generalized equation it is possible to write the following generalized algorithm:

template <class _FT, class _TT>
_TT scale_value(_FT value, _FT lowestFrom, _FT highestFrom, _TT lowestTo, _TT highestTo)
{
  return ((highestTo - lowestTo) * 
    (value - lowestFrom) / (highestFrom - lowestFrom)) + lowestTo;
}

Results

Here is some code to test the algorithm:

template <class T>
class range
{
public:
  range(T l, T h) : low(l), high(h){}

  T low;
  T high;
};

template <class _FT, class _TT>
_TT scale_value(_FT value, _FT lowestFrom, _FT highestFrom, _TT lowestTo, _TT highestTo)
{
  return ((highestTo - lowestTo) * 
    (value - lowestFrom) / (highestFrom - lowestFrom)) + lowestTo;
}

int main
{
  range<int> r0(std::numeric_limits<int>::lowest(), std::numeric_limits<int>::max());
  range<double> r1(std::numeric_limits<double>::lowest(), std::numeric_limits<double>::max());

  scale_value(r0.high, r0.low, r0.high, r1.low, r1.high)
}

And here is the result:

Value	lowestFrom	highestFrom	lowestTo	highestTo	Return Value
-100	-100	0	100	200	100
2147483647	-2147483648	2147483647	-1.79769e+308	1.79769e+308	1.#INF0e+000

Well something obviously went wrong. Breaking apart the equation makes it possible to understand where the calculation went wrong. The debugged values are commented underneath their lines.

template <class _FT, class _TT>
_TT scale_value(_FT value, _FT lowestFrom, _FT highestFrom, _TT lowestTo, _TT highestTo)
{
  _FT v_delta = value - lowestFrom; 
  //1.#INF000000000000 = 1.7976931348623157e+308 - -1.7976931348623157e+308

  _TT to_range = highestTo - lowestTo; 
  // -1 = 2147483647 - -2147483648

  _FT from_range = highestFrom - lowestFrom; 
  // 1.#INF000000000000 = 1.7976931348623157e+308 - -1.7976931348623157e+308
}

Now it is obvious what happened. When we subtract lowest from highest we end up with a value larger than the data type can hold.

The Fix

The solution here is to scale the from-range and to-range so that computing the total range will not overflow the data type. It would be possible to simply use an unsigned data type if only dealing with integral types, however floating-point types do not have unsigned versions. Thus it is necessary to scale the values down so that std::numeric_limits<T>::max() - std::numeric_limits<T>::lowest() will not overflow T. This can be done by simply reducing all input values by half, performing the calculations and then expanding the result by double.

template <class _FT, class _TT>
_TT scale_value(_FT value, _FT lowestFrom, _FT highestFrom, _TT lowestTo, _TT highestTo)
{
  //scale by half to account for negative and positive range being too large to represent
  const auto && fHLF = [](_FT v){ return v / long double(2.0); };
  const auto && tHLF = [](_TT v){ return v / long double(2.0); };

  auto scaledOffsetResult =
    ((tHLF(highestTo) - tHLF(lowestTo)) * 
    ((fHLF(value) - fHLF(lowestFrom)) / (fHLF(highestFrom) - fHLF(lowestFrom)))) * 2;

  return (_TT)(scaledOffsetResult + lowestTo); //seperated to prevent overflow
}

The Final Code

Running the previous code and once again failure is reached; the result is 1.#INF0e+000. This time the problem occurs when expanding the projected value, scaledOffsetResult, by multiplying by 2. If we expand the value before adding lowestTo then we can overflow the data type. The solution is to expand after adding lowestTo. The full solution thus becomes:

//scales 'value' from the range 'lowestFrom'/'highestFrom' to the range 'lowestTo'/'highestTo'
template <class _FT, class _TT,
class = typename std::enable_if<std::is_arithmetic<_FT>::value>::type,
class = typename std::enable_if<std::is_arithmetic<_TT>::value>::type>
_TT scale_value(_FT value, _FT lowestFrom, _FT highestFrom, _TT lowestTo, _TT highestTo)
{
  //scale by half to account for negative and positive range being too large to represent
  const auto && fHLF = [](_FT v){ return v / long double(2.0); };
  const auto && tHLF = [](_TT v){ return v / long double(2.0); };

  auto scaledOffsetResult =
    (tHLF(highestTo) - tHLF(lowestTo)) * 
    ((fHLF(value) - fHLF(lowestFrom)) / (fHLF(highestFrom) - fHLF(lowestFrom)));

  return (_TT)(scaledOffsetResult + lowestTo + scaledOffsetResult); //seperated to prevent overflow
}

Effective Modern C++ Review

Sun, 17 Aug 2014 22:36:30 +0000

Let me get this out of the way, I recommend this book. I don’t have a rating system but will go ahead and rate the book eight fruit doves out of nine fruit doves.

The Review

Scott Meyers, the author of “Effective Modern C++”, is one of the foremost experts on the C++ programming language. In his latest book, he follows the format of his previous “Effective …” books by focusing on how to take full advantage of the language. This is not a book that should be used to learn C++. In fact, it would be impossible to learn C++ with this book because it only covers a small but specific subset of the C++ programming language. Rather, “Effective Modern C++” gives you the details about how to use the features of C++11/14 to improve your code. Scott Meyers covers details of the language that are not covered anywhere else. And boy does he cover them… Want to know every reason why scoped enums are “preferable” to unscoped enums? Scott Meyers has you covered, chapter 3 item 10, and seven detail filled pages about any circumstance you could encounter while using scoped enums. This book covers details that you could only get from a true master of the language and for that reason I fully recommend it.

Preview

Here is the current table of contents, to give you a taste of the topics covered in the pre-release version of the book:

Update

Scott Meyers has provided an update on the book. With the nearly finalized TOC. It looks like there are a lot of new topics compared to the previous release. I haven’t reread the book but, certainly will as soon as possible.

The Latest Table of Contents

CHAPTER 1 Deducing Types

Item 1: Understand template type deduction.
Item 2: Understand auto type deduction.
Item 3: Understand decltype.
Item 4: Know how to view deduced types.

CHAPTER 2 auto

Item 5: Prefer auto to explicit type declarations.
Item 6: Use the explicitly typed initializer idiom when auto deduces undesired types.

CHAPTER 3 Moving to Modern C++

Item 7: Distinguish between () and {} when creating objects.
Item 8: Prefer nullptr to 0 and NULL.
Item 9: Prefer alias declarations to typedefs.
Item 10: Prefer scoped enums to unscoped enums.
Item 11: Prefer deleted functions to private undefined ones.
Item 12: Declare overriding functions override.
Item 13: Prefer const_iterators to iterators.
Item 14: Declare functions noexcept if they won’t emit exceptions.
Item 15: Use constexpr whenever possible.
Item 16: Make const member functions thread-safe.
Item 17: Understand special member function generation.

CHAPTER 4 Smart Pointers

Item 18: Use std::unique_ptr for exclusive-ownership resource management.
Item 19: Use std::shared_ptr for shared-ownership resource management.
Item 20: Use std::weak_ptr for std::shared_ptr-like pointers that can dangle.
Item 21: Prefer std::make_unique and std::make_shared to direct use of new.
Item 22: When using the Pimpl Idiom, define special member functions in the implementation file.

CHAPTER 5 Rvalue References, Move Semantics, and Perfect Forwarding

Item 23: Understand std::move and std::forward.
Item 24: Distinguish universal references from rvalue references.
Item 25: Use std::move on rvalue references, std::forward on universal references.
Item 26: Avoid overloading on universal references.
Item 27: Familiarize yourself with alternatives to overloading on universal references.
Item 28: Understand reference collapsing.
Item 29: Assume that move operations are not present, not cheap, and not used.
Item 30: Familiarize yourself with perfect forwarding failure cases.

CHAPTER 6 Lambda Expressions

Item 31: Avoid default capture modes.
Item 32: Use init capture to move objects into closures.
Item 33: Use decltype on auto&& parameters to std::forward them.
Item 34: Prefer lambdas to std::bind.

CHAPTER 7 The Concurrency API

Item 35: Prefer task-based programming to thread-based.
Item 36: Specify std::launch::async if asynchronicity is essential.
Item 37: Make std::threads unjoinable on all paths.
Item 38: Be aware of varying thread handle destructor behavior.
Item 39: Consider void futures for one-shot event communication.
Item 40: Use std::atomic for concurrency, volatile for special memory.

CHAPTER 8 Tweaks

Item 41: Consider pass by value for copyable parameters that are cheap to move and always copied.
Item 42: Consider emplacement instead of insertion.

A Wild Logo Appears

Sat, 09 Aug 2014 12:15:00 +0000

A huge thanks to Veronica Beley for designing an amazing logo for this site! She took my design requirements and created something super original and amazing!

The Design

I was overjoyed when I saw the designs. Please check her work out. Don’t be hesitant to hire her for your design projects!

Easy Timing

Sun, 03 Aug 2014 19:37:00 +0000

Timing in C++ can be tedious. There are so many different ways to accomplish stop-watch functionality, all of which involve C functions. Thankfully, C++11 introduces a brand new header that makes it easy to time things accurately with very little code. C++11 once again saves the world from unclear C-style code, and provides a great specification for handling time in many different ways.

The old C-style way of timing something:

clock_t begin = clock();
// task goes here...
double duration = (double)(clock - begin) / CLOCKS_PER_SEC;

The code isn’t hard, but it does lack clarity. Thankfully, C++11 introduces a better way to manage date and time with the <chrono> header. <chrono> is best understood by starting with the building block of measuring time, the clock. And amazingly we are given more than a single choice of clocks. The following clocks are available in <chono>:

std::chrono::high_resolution_clock
- a clock with the smallest tick period available.
- tick rate may change (depends on the value of is_steady)
std::chrono::system_clock
- a clock that represents the system real time wall clock
- tick rate may change (depends on the value of is_steady)
std::chrono::steady_clock
- a monotonic clock, meaning that time can only increase
- tick rate doesn’t change

We will look at why we need three different clocks later. For now, it is important to note that only std::chrono::steady_clock is guaranteed to give a stable tick rate. When comparing times, a stable tick rate is exactly what is needed to get a reliable assessment of how various durations relate.

Timing Things

Let’s say you want to compare the performance of two algorithms. You run your timing algorithm using std::chrono::system_clock which on your system defines is_steady as false, and these are your results:

Unstable Tick Rate

Algorithm	Timed Duration	Real Duration
algo_1	3000 milliseconds	3000 milliseconds
algo_2	3400 milliseconds	2000 milliseconds

Wow, something went seriously wrong here. We timed “algo_2” as being slower than “algo_1”, but in reality it was 1/3 faster. How did this happen? Well, in the middle of processing “algo_2”, the system changed the clock of the cpu, perhaps for thermal reasons, and with that change the tick rate changed as well. Each tick started happening faster than when we ran “algo_1”. Over the long run, the system will be able to adjust the tick rate with the changes in clock speed, and can provide an accurate wall clock time. But over small intervals, when comparing two durations, a changing tick rate will not give us a stable result.

Here are the results when using std::chrono::steady_clock which will always, on any system, define is_steady as true:

Stable Tick Rate

Algorithm	Timed Duration	Real Duration
algo_1	3050 milliseconds	3000 milliseconds
algo_2	2033 milliseconds	2000 milliseconds

What happened this time? We correctly timed “algo_2” as being 1/3 faster than “algo_1” however, the timed duration no longer matches the real duration. This time, the tick rate never changes, however it is off by a very small amount so the timed duration slowly drifts away from the real duration. When comparing times, the real duration isn’t that important, we care more about the accuracy of times when compared to each other. A stable tick rate gives us accuracy when comparing durations.

Durations

C++11 also introduces std::chrono::duration. Duration is simply a value (count) and ratio that represents fractions of a second. You can create any duration you want.

using namespace std::chrono;
duration<int,std::ratio(60 * 60 * 24, 1)> day(1);
cout << "The number of seconds in a day: " 
	<< duration_cast<duration::seconds>(day).count();

There are also the following built-in durations:

std::chrono::nanoseconds
- duration</signed integer type of at least 64 bits/, std::nano>
std::chrono::microseconds
- duration</signed integer type of at least 55 bits/, std::micro>
std::chrono::milliseconds
- duration</signed integer type of at least 45 bits/, std::milli>
std::chrono::seconds
- duration</signed integer type of at least 35 bits/ >
std::chrono::minutes
- duration</signed integer type of at least 29 bits/,std::ratio<60»
std::chrono::hours
- duration</signed integer type of at least 23 bits/, std::ratio<3600»

The Code

The Full Solution

Even with the new <chrono> header in C++11, similar to C-style time measuring, we still need clunky code blocks to wrap the things we want to time.

using namespace std::chrono;
auto begin = steady_clock::now();
// task goes here...
auto duration = begin - steady_clock::now();

The C++ version is a little bit simpler than in C, but I think it is important to explore possible ways to engineer an easier and clearer way to time code.

template<class D = std::chrono::nanoseconds, class F>
inline D time_it(F && f)
{
	using namespace std::chrono;

	auto begin = steady_clock::now();
	f();
	auto end = steady_clock::now();
	return duration_cast<D>(end - begin);
}

Leveraging templates, I have built the time_it function which takes a lambda expression and wraps it in a timing block. Additionally, time_it will cast the result to the passed in duration, defaulting to std::chrono::nanoseconds. This solution gives us two advantages over the previously presented solutions. First, we have removed tedious timing code from our function. Second, clock choice and duration interval are now independent of each other. So even if a clock defaults to nanoseconds, it is possible to get results in seconds.

The Output

Here is a simple test of the new time_it function.

using namespace std::chrono;
auto result = time_it < milliseconds >([]{ 
	for (size_t i = 0; i < 100000 * 1000; ++i) 
		std::pow(i, 2); });
std::ofstream outfile("outfile.txt");
outfile << "milliseconds: " + std::to_string(result.count());

milliseconds: 7939

Success! I hope you have learned how to more effectively and accurately measure time in C++11.