The C Programming Language – Part 8 – cpp

The C Preprocessor is just about essential. It’s probably possible to write C without it, but I wouldn’t want to try.

The preprocessor supports:

• file inclusion – recursively
• conditional compilation of chunks of text
• macro definitions and usage
• some handy ‘constants’

The preprocessor is covered lightly in chapter 4 of K&R and tersely but comprehensively in Appendix A12.

I will cover the easy parts first – leaving macros for last.

CPP Directives

All preprocessor directives begin with #.  This makes them easy to spot.

In contrast, macro usage is not distinguished in any syntactic way. That can be a problem because it is not possible to distinguish between a function call and a parameterized macro expansion from syntax alone. More below.

File Inclusion

  #include 
  #include "filename or relative path"
  #include MACRONAME

for example

  #include <stdio.h> /* searches system directories */
  #include "foo.h"   /* searches locally first, then system directories */
  #include <sys/fcntl.h> /* searches 'sys' subdir of system directories for 'fcntl.h' */
  #include FOO /* expands FOO as a macro then does one of the above - FOO should include " or <> delimiters */

Just remember that file inclusion is recursive, so if foo.h includes some files, then they will be included as well.

This is pretty simple except that the recursive expansion will often result in files being included multiple times. You must protect against this by creating a conditional compilation guard clause – explained below.

Conditional Compilation

The conditional compilation directives are

  #if numeric-expression   /* true if 'numeric-expression' is not 0 */
  #elif numeric-expression /* true if 'numeric-expression' is not 0 */
  #endif

  #ifdef macro-name      /* true if 'macro-name' is defined */
  #ifndef macro-name     /* true if 'macro-name' is NOT defined */

These guys do what you expect them to do.

The define preprocessor operator takes a macro name as an argument and evaluates to TRUE or FALSE (non-zero or 0) if the macro is defined or not. Thus ifdef and ifndef can be written as

  #if defined macroname /* equivalent to #ifdef macroname */
  #if ! defined macroname /* equivalent to #ifndef macroname */

Guarding against multiple inclusions of files is simple:

• enclose all the code in a conditional statement dependent on a unique macro name
• define the unique macro name inside the conditional

The pattern is

  #ifdef __FOO__
  #define __FOO__
   important stuff
  #endif /* __FOO__ */

I like to tack a comment on the terminating #endif so I know what’s being terminated. This is a convention which always works. Some compilers allow you to tack the actual macro name w/o enclosing it in comment delimiters, but I’m old fashioned.

Macros

Finally we get to macros. And this can get a bit weird.

Simple things first: constants. C doesn’t support named constants. Instead cpp supports non-parameterized macros which look like comments but aren’t.

  #define FOO "foo is not a real word"
  #define PI 3   /* in keeping with some arcane, misguided law */

When I write some C which includes the identifier FOO, the compiler never sees FOO. What it sees is the string which FOO is defined as. The preprocessor re-writes the text by substituting the string “foo is not a real word” (including the quote marks) for the token FOO.

This is distinctly different from a constant in – say – Ruby – which the ruby compiler does see and evaluate.

I’m going to repeat this again: The Compiler NEVER sees macros. Macros are evaluated to text strings and the text replaces the macro call.

Macros can also be parameterized – and this is where things get interesting – in the sense of ‘gee, that’s an interesting looking thing in the soup – what is it?’

  #define muller(x, y) x * y

  foo = muller(1, 2);  /* expands to foo = 1 * 2; */

  foo = muller(1 + 2, 3 - 5); /* foo = 1 + 2 * 3 - 5; */

  foo = muller(foo++, foo + 1); /* foo = foo++ * foo + 1; */

Here we have an innocent little macro to replace simple multiplication of two quantities by a silly name. It works in the simple case, but for the second two – not so much.

The right way to write this is:

  #define muller(x, y) (x) * (y)

  foo = muller(1, 2);  /* expands to foo = (1) * (2); */

  foo = muller(1 + 2, 3 - 5); /* foo = (1 + 2) * (3 - 5); */

  foo = muller(foo++, foo + 2); /* foo = (foo++) * (foo + 2); */

The inclusion of parentheses fixes the problem with second case, but the third case – well, you shouldn’t write this code this way anyway because you don’t know the order of evaluation of foo++ and foo on the right side of the assignment.

There are more rules – but this is the nut of writing macro definitions.

Just remember that macro expansion results in text which is fed to the compiler and make sure the text is completely unambiguous. Then everything will be fine – mostly.

When in doubt, compile with an option which produces the macro expansion and read what you get. (that’s the -E option for gcc)

Predefined Constants

There are a bunch of predefined constants which are very handy for writing debugging code. You use them like any other variable and the preprocessor evaluates them.

Here are two of my favorites:

• __LINE__ – the current line number
• __FILE__ – the path to the current file

As usual – go to K&R for a complete list – as of the original ANSI spec.

And this is it . . .

This is about it for this introduction to the C Programming Language.

Just a reminder – this series is intended to be a quick start to actually learning the language. I personally like working through K&R for that – the examples and problems are quite good.

For a more definitive – and relatively affordable – language definition, take a look at C – A Reference Manual by Harbison and Steele.

The C Programming Language – Part 7 – Misc

This is kind of a ‘tidy up some loose ends’ note.

typedef

typedef allows you to ‘define a type’ – which sounds more glamourous than it really is.  This has – in my opinion – both advantages and disadvantages.

The primary advantage is that it allows you to create semantically significant type names.

For example

typedef int bool_t;

creates the type bool_t which can be used to declare variables which will be used a booleans rather than normal ints. This is a good thing, not only for semantic readability of your code, but it also enhances readability. You might want to port your program to some obscure 8 bit processor which simulates int variables in software (say an Apple II which used the Mostec 6502, 8-bit beast) and so by making your bool_t a char type, you could make your program run faster.

Other creative uses of types and portability enhancements are easy to find – once you start looking.

On the other hand – in my opinion, but not the opinion of all that many other people – using typedef for struct and union types is obfuscating. My preference is to see these multi-field types declared explicitly so that when I see a type declaration of the form

v_name foo;

I know that v_name is a primitive type, not a structure or union. But that’s both a matter of taste and context. If you are working in some OS code and see something declared as proc_t you can pretty well guess that it is a procedure structure and it’s really a struct defined in some header file someplace. If that doesn’t occur to you, then you’re probably pretty new to OS internals.

One additional advantage of using typedef is that you now are expanding the vocabulary of the compiler so it can (or at least should) be able to do better type checking – thus being able to alert you to switching the order of parameters in a function call or some other bazaar thing you didn’t really mean to do.

The third reason to use typedef is to save typing – in the same way that #define-ing constants does. Consider this example from K&R – which would your rather do?

   typedef int (*PFI)(char *, char *);
    PFI foo;
    void call_foo(PFI foo_to_call);

or

    int (*)(char *, char *) foo;
    void call_foo(int(*)(char *, char*) foo_to_call);

bitfields

bitfields are a means of packing very small integers into bigger integers. They are actually a little strange in that the actual manner of stuffing these little guys into a word is implementation dependent – which means that – at best – it is either undefined or at most marginal.

The syntax is an extension of struct.

  struct {
    unsigned int foo_1: 1; /* an unsigned int 1 bit wide */
    unsigned int foo_2: 2; /* an unsigned int 2 bits wide */
    unsigned_int: 3; /* allocates 3 bits of padding; not ccessible */
    unsigned int foo_bar: 4; /* an unsigned int 4 bits wide */
    unsigned int: 0; /* force next field aligned to word boundary */
    unsigned int foo_foo: 3; /* an unsigned int 3 bits wide */
  }

Notice that the only difference between regular struct and a bitfield is the colon (:) followed by the field width.

bitfields can only be ints. Both unsigned and signed are legal, but I think it’s best to always declare them as unsigned. If you use a signed int on a 1 bit field, you have a problem knowing if its value is 0, 1, or -1 – because that (logically) should depend on the hardware architecture of the machine. Just avoid the problem and always declare them as unsigned int.

Why would you want to do that? [skip if you already know the answer]

In operating systems space is often at a premium (or at least used to be) and there were lots of flags which are represented as small integers or actual bit positions – such as 0×01, 0×02, etc. You see this sort of thing if you use the low level I/O facilities of C – constants which can be arithmetically OR’ed together to create a mask or set permission on an operation.

So it makes sense that C would support something like this.

The rub is that, at the hardware level, the positions of the bits and which end of the byte is numbered 0 and etc etc etc, varies all over the place. So C makes the decision that it will not get involved.

In short - bitfields can be used reliably within an architecture, but not between architectures. [I wouldn't trust them between alternate compilers, but then I'm paranoid]

You don’t have to use them, but you do need to understand them.

For alternatives – take a look at preprocessor macros and enum variables.

Type Qualifiers – const, volatile and register

I’ve previously covered both const and  volatile, but this post needs some filler, so I’m going over them again.

const is a promise that you will not change the value of a variable. This is a great bug catching tool because the compiler can tell if a variable which is qualified with const occurs on the left of an assignment (i.e. is an l-value) and will complain about it. You just have to use them.

volatile tells the compiler that the variable should not be optimized because it can be changed externally to the program. This sort of thing comes up in threaded code, device drivers, shared memory with cooperating processes, etc. The idea is that the compiler is not supposed to notice that we never change the value and replace it with a constant or pull it out of a loop or some such.

A real exploration of the use of volatile is way beyond anything I’m planning to do here. Just remember – if you’re writing some gnarly stuff where variables can be changed by other processes or threads, thereby setting up race conditions - volatile is your friend.

register is a hint that the variable is used a lot. This used to be a good idea because it indicated to the compiler that the variable should be kept in a register rather than memory. Modern optimizing compilers have made this qualifier useless. But it doesn’t seem to do any harm, so it might make sense to use it as a form of documentation.

Next time: the C Preprocessor

The C Programming Language – Part 6 – Pointers Again

pointer – a scalar variable containing a memory address and having an associated data type.

pointers are one of the things which give C a bad reputation.

Why?

Even though pointers are very simple in concept and easy to use, they provide almost unrestricted access to the memory a program is using. This is potentially dangerous when used by malicious users or idiots. pointers require great care to use safely – but, like guns, knives, and automobiles, they are necessary for some tasks. Unlike guns, knives and automobiles, we don’t have authorities issuing licenses for pointer use.

Properties

Here’s some code from last time:

  struct foo {
    int x;
    float y;
  };

  char *cp = 0x0;
  int i = 12;
  int *ip = &i;
  struct foo *foo_ptr;

I intentionally assigned cp the value 0 so that it gives the program a means to access base memory. Base memory is usually protected, read-only memory. On some machines it is (or can be) restricted to only holding instructions for execution. This sort of restriction is implemented in hardware so that if an illegal access is detected the machine traps to an interrupt which invokes some operating system code which in turn kills of the program which is running and returns it some sort of error code. This is usually called a segmentation fault because the segment of memory was accessed in an illegal way (a fault). The response of the program to this exception is usually to dump the contents of memory for the program so the hapless programmer can try to find out what happened. This is called a core dump in homage to core memory which is an ancient means of implementing persistent computer memory using little bitty fero-ceramic donuts and wires. But enough history.

The point is: if you have a pointer you can attempt to modify anything in your program. It may or may not work.

So why have something so dangerous? And why do you need to understand pointers to write C? Can’t you just avoid them?

Well, no, you can’t avoid them. For example:

#include <stdio.h>
#include <stdlib.h>

struct foo_struct {
  int x;
  float y;
};

void prt_foo_struct(struct foo_struct);
void prt_foo_struct_ptr(struct foo_struct *);

main()
{
  struct foo_struct foo = { 2, 3.6 };

  prt_foo_struct(foo);
  prt_foo_struct_ptr(&foo);
}

void prt_foo_struct(struct foo_struct foo) {
  printf("prt_foo_struct:\n");
  printf("foo: %x\n", foo);
  printf("foo: %d, %f\n", foo.x, foo.y);
}

void prt_foo_struct_ptr(struct foo_struct *foo_ptr) {
  printf("prt_foo_struct_ptr:\n");
  printf("foo_ptr: %x\n", foo_ptr);
  printf("foo: %d, %f\n", foo_ptr->x, (*foo_ptr).y);
}

First thing to remember is that in C all parameters of functions are call by value. Which means that the value of the variable is copied, pushed on the stack and that copy is available to the function as an automatic, but initialized variable.

In the case of prt_foo_struct the parameter is a struct foo, so the entire structure is copied and pushed on the stack. This has the advantage of allowing the receiving function to mutilate, spindle, fold and decimate it’s parameter without recrimination.

In the other case, prt_foo_struct_ptr receives a pointer to the structure. This function cannot be so carefree in its manipulation of the structure, but the pointer is smaller than the structure. In fact pointers are almost always the same size irrespective to what they point to. This is both memory and time efficient.

When you run it you get:

prt_foo_struct:
foo: 2
foo: 2, 3.600000
prt_foo_struct_ptr:
foo_ptr: 65a2cad0
foo: 2, 3.600000

Let’s go through this line-by-line – ignoring the function name announcements.

• foo: 2 – this is the result of printing the struct foo data under the assumption that it’s an unsigned integer. gcc generates a warning about this when compiling this program: “ptr_1.c:22: warning: format ‘%x’ expects type ‘unsigned int’, but argument 2 has type ‘struct foo_struct’” – which is really nice to have and one of the advantages ANSI C brought us over ‘traditional K&R C’.
• foo: 2, 3.60000 – is pretty straight forward. Notice that the fields of foo are referenced using the ‘dot’ notation which is correct for struct variables.
• foo_ptr: 65a2cad0 – this is the value of the pointer which is passed in the call ‘prt_foo_struct_ptr(&foo);’. This hexadecimal number is the actual address of ‘foo’ in the function main and is on the calling stack of main.
• foo: 2, 3.6000 – is the same result – which makes sense because we are printing out the values of the struct – which must be the same as the copy passed to the other function. Again, not the use of the arrow syntax which is short hand for (*foo_ptr).x and (*foo_ptr).y. [I wrapped the dereference operation in parentheses to make the order of application crystal clear. It's also syntactically necessary because '.' binds tighter than '*' - but that's an issue for precedence table aficionados]

Now lets apply this to the real world of operating systems programming.

Operating systems contain all kinds of structures – often big ones – to keep track of process state, file state, file system state, network connections and communication buffers, etc etc and on and on and on. These things need to be passed to functions and – as they say – time is of the essence because every wasted nanosecond in the OS is resources lost to ‘doing real work’.

So do you want to be copying huge structures all the time or do you want to be passing pointers around?

The obvious answer is pointers.

Arrays

Let’s take a side step and talk about arrays for a moment.

An array is just a sequence of identical data items. For example:

  #include <stdio.h>

  struct foo_struct {
    int x;
    float y;
  };

  void prt_foo_struct_ptrs(struct foo_struct *, int);

  main()
  {
    struct foo_struct foo_ar[12];

    foo_ar[0] = (struct foo_struct){ 2, 3.6 };
    foo_ar[1] = (struct foo_struct){12, -3.4 };
    prt_foo_struct_ptrs(foo_ar, 2);
  }

  void prt_foo_struct_ptrs(struct foo_struct foo_ptr[], int len)
  {
    printf("prt_foo_struct_ptrs\n");
    while (len-- > 0) {
      printf("foo_ptr: %x\n", foo_ptr);
      printf("foo element: %d, %f\n", foo_ptr->x, foo_ptr->y);
      foo_ptr += 1;
    }
  }

Here we have an array of struct foo_struct which is passed to a function by passing a reference to the first element of the array.

The line ‘struct foo_struct foo_ar[12];’ simply allocates the automatic variable foo_ar on the stack of the main function. The ‘[12]‘ tells the compiler to reserve memory for 12 struct foo_struct variables in contiguous memory.

The two assignment statements stuff values into the first two locations in the array foo_ar. (array indexes start at 0 because they are ‘really just memory offsets from a base address’ rather than some mathematical concept). Notice that the right hand side (r-value) of each assignment is cast as a struct foo_struct. This keeps the compiler from complaining and ensures that the values are put away in the right places. (normally I write out the field assignments because I’m untrusting, but – as usual – your mileage may vary)

Running this program generates this output:

prt_foo_struct_ptrs
foo_ptr: 64d4fa68
foo element: 2, 3.600000
foo_ptr: 64d4fa70
foo element: 12, -3.400000

So let’s look at the function prt_foo_struct_ptrs for a minute.

It looks like prt_foo_struct_ptrs is passed an array and an int, but it’s really getting a pointer to an array and an int. Remember: when you pass an array to a function, you implicitly get a pointer – not a copy of the array, as you do in a structure. This is a conscious design choice in the language: it allows us to write functions which work on arbitrary length arrays. As an example, you might be implementing some vector stuff and need to add vectors – so rather than have functions for each length, you would rather have a single function which takes two vectors and a length.

Notice that the function doesn’t really have a clue how big the array it’s passed is. In fact, it doesn’t even really know that it’s being passed an array. All it knows it that the first argument is a pointer to a foo_struct and the second argument is an int.

It assumes that there are len contiguous struct foo_struct things out there which it can access. So it just accesses them and prints out their field values.

I could have (unintentionally, of course) lied and called it with a value of 32 for len and it would have happily accessed 32 struct foo_struct chunks of memory – well beyond the amount of data allocated for foo_ar in the main function. This could have been benign or disastrous – but it would have been legal and would have been executed.

Summary – for now

If you write C, you have to understand pointers and know when they occur both explicitly and implicitly.

To go into more depth, study K&R Chapter 5.

Next Time we’ll talk about typedefs, bit-fields and complex type declarations.

The C Programming Language – Part 5 – structs, unions and simple pointers

struct and union are complex data structures – meaning that they can contain multiple values.

pointers are memory location addresses with associated data types. We’re just going to introduce them here and go into them in more depth next time.

A struct is:

• a record with a name and one or more named fields
• or an Object without any methods (other than accessors)
• or an SQL table with one or more fields

The fields are accessed using a ‘dot’ notation.

For example:

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <memory.h>

  struct foo {
    int x;
    float y;
    char str[10];
  };
  struct foo foo_struct;

  struct foo init_foo(int x, float y, const char* str)
  {
     struct foo foo_tmp;
     int str_len = strlen(str);

     foo_tmp.x = x;
     foo_tmp.y = y;
     memcpy(foo_tmp.str, str, str_len >= sizeof(foo_tmp.str) ?
       sizeof(foo_tmp.str) - 1 : str_len);
     foo_tmp.str[sizeof(foo_tmp.str) - 1] = '';

     return foo_tmp;
  }

  main()
  {
    struct foo foo_local;

    foo_local = init_foo(12, -23.6, "a long string");

    printf("foo_local.x: %d\n", foo_local.x);
    printf("foo_local.y: %f\n", foo_local.y);
    printf("foo_local.str: %s\n", foo_local.str);
  }

So the important things to notice about a struct are:

• It contains named fields
• Each field has it’s own, unique storage.

You might also want to note that I used the sizeof operator to have the compiler compute the size of the str field in struct foo. This makes the program more robust and portable. It also doesn’t cost anything at run time, because the compiler (who knows all) figures it out while compiling the program and replaces it with the actual value.

Another thing to note about struct types is that there can be holes in them.

Huh?

When working in C you need to be aware of boundary alignment issues which come up because certain types of data may only start on memory addresses which are even or divisible by 4 or . . . well, you get the idea.

In struct foo I have allocated 3 types of data. The struct will begin on a nice boundary, but the float y may have some unused space between where it begins and where int x ends. Or it may not – it depends on the size of these datum and the architecture of the machine the code is running on.

If I wanted to guarantee some wasted space, I would put a single character variable between int x and float y.

unions

A union looks just like a struct – except for beginning with the key word union. All the rest of the syntax is the same.

The difference is that every field in a union starts at the same memory location!

For example:

  #include <stdio.h>
  #include <stdlib.h>
  #include <memory.h>
  #include <string.h>

  union anything {
    char c;
    int  i;
    float f;
    char ca[10];
    char *cp;
  };

  main()
  {
    union anything foo;

    strcpy(foo.ca, "short str");
    printf("foo.c: %c - size is %d\n", foo.c, sizeof(foo.c));
    printf("foo.i: %d - size is %d\n", foo.i, sizeof(foo.i));
    printf("foo.f: %f - size is %d\n", foo.f, sizeof(foo.f));
    printf("foo.ca: %s - size is %d\n", foo.ca, sizeof(foo.ca));
    printf("foo.cp: %x - size is %d\n", foo.cp, sizeof(foo.cp));
  }

Here’s the output:

  foo.c: s - size is 1
  foo.i: 1919903859 - size is 4
  foo.f: 4741964077099963063589868142592.000000 - size is 4
  foo.ca: short str - size is 10
  foo.cp: 726f6873 - size is 8

Why would you ever do that?

Here’s one reason: Suppose you got some data from some strange place and needed to translate it into the binary format of your computer hardware [think networking code between two computers of different architectures]. The union allows you to reinterpret the bit patterns of a chunk of memory using any of several different data types.

This is different from type conversion or type coercion – where the bits are assumed to mean something known in the architecture of your machine and that meaning is translated to a different data type.

In my experience unions don’t come up all that often – but when you need them, they are there.

Pointers

Pointers are really simple – in concept:

A pointer is a scalar variable which contains a memory address and has an associated data type.

Here are some examples:

  struct foo {
  int x;
  float y;
};

char *cp = 0x0;
int i = 12;
int *ip = &i;
struct foo *foo_ptr;

In the example the three variables cp, ip and foo_ptr are all pointer variables. I’ll go through each one separately.

• cp – is a character pointer. It is initialized to the value 0×0 – which should be the base memory location of your program. If you want to create havoc, you can use cp to write something to location 0×0 by simply assigning the memory location cp points to a value, as in ‘*cp = “foobar”;’ This should cause a segmentation fault and subsequent core dump (don’t worry about  it if you don’t know what that means – just ask somebody when it happens)
• i – is an integer variable. The definition of i reserves some storage and initializes the value of that storage to 12
• ip – is an integer pointer which is initialized to the address of the variable i. This gives us two ways to access the variable i – through i or by dereferencing the pointer ‘ip’ using the syntax ‘*ip’.
• foo_ptr – is just a pointer to a struct of type foo. It is not initialized, but by  now you should understand that if foo_ptr is assigned a value, then the memory at that location can be accessed as though it were at struct foo no matter what it ‘really is’.

That’s enough about pointers for now. Next time we’ll explore them in more depth and demonstrate how to get into real trouble and talk a bit about buffer overflows and casts.

The C Programming Language – Part 4 – Primitive Types

Primitive types hold one value – so this excludes such things as arrays, strings, unions, structs, objects, or anything else which is complex.

Technically, pointers only hold a single value – so in that sense they are primitive. But they are more complex than integers and other typical primitive types, so we’ll discuss them separately and not consider them to be primitive. As usual, your mileage may vary.

So – with the qualifiers out of the way – what are we talking about.

Numbers

C supports several different kinds of numbers

• integers – that is whole numbers without fractions.
• floating point numbers - computer-ish approximations of real numbers.
• enumerators or small sets of numbers - these guys are more of a convenience. They let you assign symbols to distinguished things and use the type system to help keep from messing up.

integers

Integers come in a bewildering variety of sizes and two ranges.

First, the ranges are signed and unsigned.

signed integers can be positive and negative. The underlying number representation depends on the machine – but is usually 2′s complement anymore. I think – or rather hope – that the old 1′s complement machines are extinct.

unsigned integers are straightforward: they start a 0 and count up to the maximum representable integer for the given integer size.

The sizes are:

• char - pretty much your standard,  8-bit byte.
• int - ‘the native integer size of the machine’ which is usually 16, 32, or maybe 64 bit’s wide. Although I’ve seen machines with 15, 16, 18, and 36 bit integers. There have been other widths – but int is whatever the ‘native width’ is.
• short int – is usually 16 bits wide, but y.m.v.
• long int - is usually 32 bits, but again . . .
• long long int - is something I’ve seen around and it’s definitely bigger than long int’.

All integer definitions/declarations default to signed. Thus a char is a data type which ranges between -128 and 127 (on a 2′s complement machine), whereas unsigned char ranges between 0 and 255 and is probably the 8-bit byte you’re probably looking for.

Similarly, int, short int, and long int are all signed, whereas unsigned int, unsigned short int, and unsigned long int are not.

As a convenience, int can be dropped for short and long variables. That is unsigned long means the same thing as unsigned long int.

Floats

Floats are simpler.

They generally come in two sizes: float and double. [you can get bigger floats. left as an exercise for the reader if you need them]

C implicitly converts float variables to double values in a variety of circumstances – such as when used as arguments to functions. This is convenient and generally doesn’t do any harm.

Enums

An enum is really an unsigned integer ranging over a fixed, small set of declared values.

For example:

  enum my_bool = { False = 0, True };
  enum old_geek_food = { Pizza, Bananas, Coffee, Fries };

In this example my_bool can take on values of False and True. False is initialized to 0 and the compiler implicitly sets True to 1 (one more than the previous value). This allows us to compare enum values and convert them to integers. old_geek_food is really the values 0, 1, 2 and 3, but with nice names.

This makes it easier to write programs with mnemonically meaningful values rather than stupid little numbers – which we used to do (when programming by candle light with mouse driven computers).

See K&R for a more thorough discussion.

Constants

Constants are just that: constants. They are pretty much what you are used to, except C is strict about string and character constants.

string literals are enclosed in double quote marks. Always. No exceptions.

string literals can be concatenated by just writing them, as in:

  "part one" "part two"

which results in the literal

   "part onepart two"

which probably not what you wanted

All the nice backslash escapes you’re used to originated in C (at least as far as I know), so you can use them in strings.

You can also use them in character constants.

character constants are things like ’0′ and ‘\x02′ and ‘\n’. They are all one (1 only) character long and are not strings!!!!!

In fact – C strings are really arrays of characters which terminate in a 0×00 byte. As a result, I can build a string from character constants.

Again – read K&R and do some experiments.

Adventures in Cloning ActiveRecord Instances

The short version: DON’T clone – dup instead!

Now here’s the longer version.

The Problem

The problem with ActiveRecord objects is that ActiveRecord is an
Object-Relational-Database implementation. In addition to providing
a map between Ruby objects to Database Tables, ActiveRecord also
maintains coherence between distinct in-code object instances which
reference the same database entry.

This is a good thing and generally what you want. Let’s say
you pull a record from the data store in one part of you
program and – for reasons only known to you and your closest relatives
- you create another instance someplace else which you also pull
from the data store.

Now you modify one of those instances. You really want the modifications
you make to magically be applied to the other instance you’ve
created.

ActiveRecord appears manages this all behind the scenes – at least according
to my testing.

Unfortunately this can collide with what you want if you want to make a
copy of an object and then manipulate it.

Why would you want to do that?

In my case I was implementing some simple vector operations on an array
of data – scalar multiplication, additions, subtraction – stuff like that.
A natural (and functional programming-ish) way to do this is to create
a temporary new object to hold the results.

So for scalar multiplication I just – naively – cloned my original
object, expecting to be able to simply multiply all the associated
data (my real vector) by my scalar without munging up my original
object.

Boy did that not work.

The answer is: Use ‘dup’ instead of ‘clone’.

The rest of this is why and what’s going on under the covers.

What Ruby Does when you clone and dup

Both clone and dup give you a shallow copy of the
object instance you are replicating. The difference is between them is
technical. I pulled this from Programming Ruby 1.9 by
Dave Thomas:

• both produce a shallow copy – which means that they copy all the
  instance variables, but none of the objects they refer to.
  This isn’t generally a problem unless one of the variables
  refers to an object and, rather than replacing it, you modify one of
  that object’s instance variables. See the example below.
• clone also copies the object’s frozen and tainted state and
  any associated singleton class. If you don’t know what this means,
  then you need to read the book – but it’s a good thing.
• dup copies the tainted state, but that’s all.

Here’s an example. Foo is an object with two attributes and stubs for
specializing cloning and duping.
We create an instance and then adds a function to the instance -
creating a singleton class – and create both a clone and a dup.
Finally we modify the attributes and display the state of all three
instances.

The details are discussed below the code.

  class Foo
    attr_accessor :name, :an_int, :a_float, :a_string, :another_str

    def initialize name, a_string, another_str
      self.name = name
      self.a_string = a_string
      self.another_str = another_str
    end

    def prt
      puts name
      puts "a_string: #{a_string}"
      puts "another_str: #{another_str}"
      self.func if defined? self.func
      puts "method func not defined" unless defined? self.func
    end

    # overridable methods which are called during cloning and duping
    def initialize_clone object ; puts "in initialize_clone" ; super ; end

    def initialize_dup object ; puts "in initialize_dup" ; super ; end

    private
    def initialize_copy object ; puts "in initialize_copy" ; super ; end
  end

  # create a Foo and add a function to the singleton class
  foo = Foo.new 'foo', "abcde", 'xx-yyyy'
  class << foo
    def func ; puts 'func is defined' ; end
  end

  foo_clone = foo.clone
  foo_clone.name = 'foo_clone'
  foo_dup = foo.dup
  foo_dup.name = 'foo_dup'

  puts "after clone and dup, but no changes"
  [foo, foo_clone, foo_dup].each { |x| puts x.prt }

  foo_clone.a_string = "clone value"
  foo_clone.another_str[0,2] = 'zz-clone'

  puts "after modifying foo_clone"
  [foo, foo_clone, foo_dup].each {|x| puts x.prt }

  foo_dup.a_string = "dup value"
  foo_dup.another_str[0,2] = 'dup'

  puts "after modifying dup value"
  [foo, foo_clone, foo_dup].each {|x| puts x.prt }

Here’s the output of running this example:

  in initialize_clone
  in initialize_copy
  in initialize_dup
  in initialize_copy
  after clone and dup, but no changes
  foo
  a_string: abcde
  another_str: xx-yyyy
  func is defined

  foo_clone
  a_string: abcde
  another_str: xx-yyyy
  func is defined

  foo_dup
  a_string: abcde
  another_str: xx-yyyy
  method func not defined

  after modifying foo_clone
  foo
  a_string: abcde
  another_str: zz-clone-yyyy
  func is defined

  foo_clone
  a_string: clone value
  another_str: zz-clone-yyyy
  func is defined

  foo_dup
  a_string: abcde
  another_str: zz-clone-yyyy
  method func not defined

  after modifying dup value
  foo
  a_string: abcde
  another_str: dup-clone-yyyy
  func is defined

  foo_clone
  a_string: clone value
  another_str: dup-clone-yyyy
  func is defined

  foo_dup
  a_string: dup value
  another_str: dup-clone-yyyy
  method func not defined

Example Details

The attribute a_string is modified by replacing it’s value. This
does not spill outside of the cloned or dup’ed object

The attribute another_str is modified by actually accessing the
underlying object and using its [] method to modify the content. The
underlying object is accessed by all three instances – because the reference
was copied, not the underlying object.

The method func is added to the foo object by creating
a singleton class and putting the method in there. This method is then
callable from both foo and from foo_clone, but not from
foo_dup because it does not exist in the Foo class definition,
but only for the instance foo an its clones.

So What if That’s Not What You Want

Ruby defines 3 methods which allow you to clean up after cloning or duping
an instance. They are – not surprisingly initialize_clone,
initialize_dup and initialize_copy. Both clone and
dup call their respective initializer. Both of those initializers
simply call initialize_copy – so if you have common behavior that
you always want to do when you replicate a objects of your class,
just override initialize_copy and forget about the other two.

If you want specialized behavior depending on whether you call clone
or dup, then you have the ability to handle that by overriding the
specific initializers.

What ActiveRecord does when you clone and dup

Now we’re ready to look at what ActiveRecord does – because
ActiveRecord chooses to do different things for clone
and dup.

The first problem is figuring out what ActiveRecord is doing.
But now that we know where to look – we just need to find where
the three initialize_dup/clone/copy are and look at the code.

The fastest way I
know of to find out what’s going on is to examine the ancestors of
ActiveRecord::Base – which all models are derived from. Fortunately this
is easy in Ruby – you bring up a rails console in a rails app and run
something like:

  regx = /initialize_(dup|clone|copy)/
  ActiveRecord::Base.ancestors.each do |c|
    l = c.instance_methods(false).grep(regx) +
      c.protected_instance_methods(false).grep(regx) +
      c.private_instance_methods(false).grep(regx)
    puts "#{c}: #{l.inspect}" unless l.empty?
  end ; nil

When I run this on a Rails 3.2.7 app I get:

  ActiveRecord::Base: [:initialize_dup]
  ActiveRecord::Timestamp: [:initialize_dup]
  ActiveModel::Validations: [:initialize_dup]
  Kernel: [:initialize_dup, :initialize_clone, :initialize_copy]

We can see that ActiveRecord is doing something special for dup
but is using straight Ruby for clone. This explains the problems
I ran into because ActiveRecord seems to be going to great lengths to
maintain coherence among concurrent object instances which refer to
the same database record and all of the reference information is
copied when doing a clone.

Further – if we want to find out what is really going on when we dup
an ActiveRecord object, we know where to look.

So what is ActiveRecord doing?

ActiveRecord::Base#initialize_dup

This does all kinds of stuff. If you override initialize_dup you
should definitely call super first so that your replicate is properly
initialized. Then do whatever else you need to do – such as dup all
the records of any associations which are in the original record.

ActiveRecord::Timestamp#initialize_dup

Clears the timestamp attributes and marks them as unchanged.

In this case, initialize_dup does not call super – which
may be a bug because this stops the call propagation so that if you
override initialize_copy it will not be called.

ActiveModel::Validations#initialize_dup

Just clears the @errors instance variable and runs super if it
exists.

Summary

First – use dup not clone

Second – do all your special purpose clean up in initialize_dup
not initialize_copy because of the way
ActiveRecord::Timestamp#initialize_dup is currently coded.

Third – when you override initialize_dup, call super first
so that your instance is properly set up.

The C Programming Language – Part 3 – Scope

Scope refers to the segment of a program where an identifier has meaning.

Let’s be more specific.

C programs are written in plain text files which have a .c extension. Each of these files is referred to as a compilation unit in recognition that the file can be processed by the C compiler to generate object code - i.e. this is something which can be compiled. object code is essentially machine code, but which cannot actually run because all addresses are not defined and the bootstrap code is not in place. The process of linking or link editing combines all the object code in a program with the bootstrap code and references to libraries to create a runnable executable.

If this doesn’t make a lot of sense – just remember that – for the purposes of discussing scope we are talking about C source code files.

A source file will contain various things:

• type definitions – discussed later
• variable definitions – this defines a global variable which is available anywhere in the program (with specific qualifications we’ll get into below)
• variable declarations – which associate an identifier with a data type and indicate to the compiler that the storage fo the variable is allocated ‘somewhere else’
• function definitions – which we’ve seen before
• function prototypes – which are similar to variable declarations in that they don’t define the code in a function, but define the calling signature, return value and the identifier which names the function.
• macro directives – much later

OK – so let’s summarize:

A variable which is defined in a file (aka a compilation unit), outside of a function definition has global scope. Which means that it can be seen anywhere below it’s definition in that compilation unit and in any other context in which it is declared using the extern keyword (with exceptions described below).

Now it’s time to look at the words define and declare - as used in K&R.

• to define  a variable means to associate an identifier with both a data type and storage. Variables may only be defined once in a program – i.e. you only define a variable in one compilation unit.
• to declare a variable means to associate an identifier with a data type and to specify it’s scope. Typically this is done using by prefixing extern keyword to a variable type specification. This informs the compiler that the variable exists, has a specific data type, and that the storage is allocated elsewhere.
• All variables must be either defined or declared in order to be used in a program.

Local Variables

Variables defined within functions have local scope. These are usually automatic variables which exist only for the duration of the function call. automatic variables are created as a function – or enclosed block – is entered and evaporate as the function returns or the block is exited.

local variables mask variables at higher scoping levels as long as they exist.

For example:

  int x = 12;
  int y = 4;

  int main(int argc, char** argv)
  {
     int x = 13;

     {
       int x = 5;
       printf("x is now 5: %d\n", x);
     }

     printf("x is now 13: %d\n", x);
     printf("y has always been 4: %d\n", y);
     exit(0);
   }

So this is pretty simple – no?

Getting a little more Complex

Sometimes you want variables to be both local to functions and to retain their values between calls. This gets us into the idea of storage class.

A local variable is – by default – automatic. It is created anew each time it’s scope of definition is entered and discarded when it’s enclosing block is exited. It’s initial value is undefined - which means that it is whatever garbage which is in memory at the location on the stack it temporarily occupies. This can lead to some interesting consequences, such as:

• using a local variable before it is initialized – the compiler is supposed to catch this and warn you. But it might be a legitimate use in a device driver if the variable is somehow associated with a device register or shared memory (good trick to pull that off w/o pointers, though)
• returning the address of an automatic variable. Doing this will return the address of a chunk of the call stack which is no longer valid nor under control of the function which used it. I suppose modern compilers take all the fun out of that by warning you now, but I think there are easy ways to get around it – like saving the address in a pointer and returning it or passing it to a function which returns that value and returning that function or something more arcane.

local variables may be prefixed with the static keyword to inform the compiler to allocate permanent space for them – which is only available within the context of the function. This allows one to write iterators which can work their way through arrays because they have a local variable which retains its value between calls.

For example:

 #define FOO_SIZE 100

  struct foo_struct {
    int x;
    int y;
  } foo[FOO_SIZE];

  struct foo_struct *next_foo()
  {
    static int foo_index = 0;

    if (foo_index < FOO_SIZE) {
      return &foo[foo_index++];
    } else {
      foo_index = 0;
      return (struct foo_struct*)0;
    }
  }

But Wait – There’s Another Use for static

Sometimes you’d like to define variables and functions in a compilation unit (remember – that’s just a file) that are private to the file they are contained in. You can do this by prefixing their definition by the static keyword.

I don’t know why K&R uses static this way – but they do. I guess it was because ‘static’ is already a keyword and so ‘why not use it rather than creating another one’. In this usage it really means ‘private to this file – but “acts like” a regular global or function’

Here’s the iterator rewritten using global variables which are private to the compilation unit:

  #include <stdlib.h>

  struct foo_struct {
    int x;
    int y;
  };

  struct foo_struct *init_foo(int);
  void destroy_foo(void);
  struct foo_struct *first_foo(void);
  struct foo_struct *next_foo(void);
  static void useless_func(void);

  static struct foo_struct *foo_ptr;
  static int foo_size = 0;
  static int foo_index = 0;

  struct foo_struct *init_foo(int n)
  {
    if (foo_ptr) {
      free(foo_ptr);
    }
    foo_ptr = malloc((foo_size = n) * sizeof(struct foo_struct));
    foo_index = 0;
    return foo_ptr;
  }

  void destroy_foo(void)
  {
    if (foo_ptr) {
      free(foo_ptr);
    }
    foo_size = 0;
    foo_index = 0;
    foo_ptr = (struct foo_struct*)0;
  }

  struct foo_struct *first_foo(void)
  {
    if (foo_ptr) {
      foo_index = 0;
      return next_foo();
    } else {
      return (struct foo_struct *)0;
    }
  }

  struct foo_struct *next_foo(void)
  {
    if (foo_index < foo_size) {
      return foo_ptr + foo_index++;
    } else {
      return (struct foo_struct*)0;
    }
  }

  static void useless_func(void) { }

Notice the use of static in the definitions of each of the three variables: foo_ptr, foo_size, and foo_index. This keeps them private to this file and protects them from name collision and accidental mangling by code in other files.

The function useless_func() is also declared static. This makes it really useless because it is not visible outside of the compilation unit (remember – it’s just a file) and it isn’t called inside it.

The original fixed implementation now uses dynamic allocation of memory, so the program can pick the size of the foo_struct array at run time – as needed. I’ve also added a destructor and iteration initializer.

WARNING: this code compiles, but hasn’t been tested so if it blows up or burns your pizza, you’ve been warned and I’m not responsible.

Those are the important ones – but . . .

There are a couple of more keywords which modify the interpretation of variables. They’re pretty specialized, so you can ignore them – most of the time.

• register - the register prefix is tells the compiler that ‘this variable is used a lot, so you should try to allocate it to a register rather than storage – if you can – but if you can’t, it’s OK’. Modern optimizing compilers are pretty good – better than I am any way – so register is more of an ancient artifact than something really useful (Please add a comment if you know of a case where this is not true)
• volatile - the volatile prefix tells the compiler that ‘this variable might change when you’re not looking, so don’t assume it’s a constant just because you haven’t seen any code which changes it’. That’s necessary if you’re writing a device driver that needs to watch a memory mapped location to read something updated by external events – like a clock or an analog to digital converter output or some such.
• const - the const prefix tells the compiler that ‘this variable will not be changed’. It’s not really a constant in the sense that it’s invariant throughout the life of the program. This is really a compiler directive so that if you actually do change the variable (assign it a new value) the compiler can generate a warning that you mucked up. 

That’s it for now

Nest time we’ll do a whirl wind tour of primitive types – ints, floats, char’s, unsigned’s, enums, and anything else which just has a single value (but isn’t a pointer)