Pointer Assignment Vs Memcpy Linux

Dereferencing a null pointer is undefined behavior.

On many platforms, dereferencing a null pointer results in abnormal program termination, but this is not required by the standard. See "Clever Attack Exploits Fully-Patched Linux Kernel" [Goodin 2009] for an example of a code execution exploit that resulted from a null pointer dereference.

Noncompliant Code Example

This noncompliant code example is derived from a real-world example taken from a vulnerable version of the library as deployed on a popular ARM-based cell phone [Jack 2007]. The   library allows applications to read, create, and manipulate PNG (Portable Network Graphics) raster image files. The library implements its own wrapper to that returns a null pointer on error or on being passed a 0-byte-length argument.

This code also violates ERR33-C. Detect and handle standard library errors.

If  has the value , the addition yields 0, and subsequently returns a null pointer, which is assigned to . The pointer is later used as a destination argument in a call to , resulting in user-defined data overwriting memory starting at address 0. In the case of the ARM and XScale architectures, the address is mapped in memory and serves as the exception vector table; consequently, dereferencing did not cause an abnormal program termination.

Compliant Solution

This compliant solution ensures that the pointer returned by is not null. It also uses the unsigned type to pass the parameter, ensuring that negative values are not passed to .

Noncompliant Code Example

In this noncompliant code example, is copied into dynamically allocated memory referenced by . If fails, it returns a null pointer that is assigned to . When is dereferenced in , the program exhibits undefined behavior.  Additionally, if is a null pointer, the call to dereferences a null pointer, also resulting in undefined behavior. This code also violates ERR33-C. Detect and handle standard library errors.

Compliant Solution

This compliant solution ensures that both and the pointer returned by are not null: 

Noncompliant Code Example

This noncompliant code example is from a version of and affects Linux kernel 2.6.30 [Goodin 2009]:

The pointer is initialized to before checking if is a null pointer. Because null pointer dereferencing is undefined behavior, the compiler (GCC in this case) can optimize away the check because it is performed after is accessed, implying that is non-null. As a result, this noncompliant code example is vulnerable to a null pointer dereference exploit, because null pointer dereferencing can be permitted on several platforms, for example, by using with the flag on Linux and Mac OS X, or by using the POSIX function with the flag [Liu 2009].

Compliant Solution

This compliant solution eliminates the null pointer deference by initializing to following the null pointer check:

Risk Assessment

Dereferencing a null pointer is undefined behavior, typically abnormal program termination. In some situations, however, dereferencing a null pointer can lead to the execution of arbitrary code [Jack 2007, van Sprundel 2006]. The indicated severity is for this more severe case; on platforms where it is not possible to exploit a null pointer dereference to execute arbitrary code, the actual severity is low.

Rule

Severity

Likelihood

Remediation Cost

Priority

Level

EXP34-C

High

Likely

Medium

P18

L1

Automated Detection

ToolVersionCheckerDescription
Astrée

17.04i

null-dereferencingFully checked
CodeSonar

4.5p1

LANG.MEM.NPD
LANG.STRUCT.NTAD
LANG.STRUCT.UPD

Null pointer dereference
Null test after dereference
Unchecked parameter dereference

Compass/ROSE

Can detect violations of this rule. In particular, ROSE ensures that any pointer returned by , , or is first checked for before being used (otherwise, it is -ed). ROSE does not handle cases where an allocation is assigned to an lvalue that is not a variable (such as a member or C++ function call returning a reference)

Coverity

2017.07

CHECKED_RETURN

NULL_RETURNS

REVERSE_INULL

FORWARD_NULL

Finds instances where a pointer is checked against and then later dereferenced

Identifies functions that can return a null pointer but are not checked

Identifies code that dereferences a pointer and then checks the pointer against

Can find the instances where is explicitly dereferenced or a pointer is checked against  but then dereferenced anyway. Coverity Prevent cannot discover all violations of this rule, so further verification is necessary

Cppcheck

1.66

nullPointer, nullPointerDefaultArg, nullPointerRedundantCheck

Context sensitive analysis

Detects when NULL is dereferenced (Array of pointers is not checked. Pointer members in structs are not checked.)

Finds instances where a pointer is checked against and then later dereferenced

Identifies code that dereferences a pointer and then checks the pointer against

Does not guess that return values from , , etc., can be (The return value from is only if there is OOMo and the dev might not care to handle that. The return value from is often , but the dev might know that a specific function call will not return .)

Klocwork

2017

NPD.CHECK.CALL.MIGHT
NPD.CHECK.CALL.MUST
NPD.CHECK.MIGHT
NPD.CHECK.MUST
NPD.CONST.CALL
NPD.CONST.DEREF
NPD.FUNC.CALL.MIGHT
NPD.FUNC.CALL.MUST
NPD.FUNC.MIGHT
NPD.FUNC.MUST
NPD.GEN.CALL.MIGHT
NPD.GEN.CALL.MUST
NPD.GEN.MIGHT
NPD.GEN.MUST
RNPD.CALL
RNPD.DEREF


LDRA tool suite

9.7.1

45 D, 123 D, 128 D, 129 D, 130 D, 131 D, 652 S

Fully implemented
Parasoft C/C++test

10.3

BD-PB-NPFully implemented
Parasoft Insure++

Runtime analysis
Polyspace Bug FinderR2016aArithmetic operation with NULL pointer,
Null pointer,
Use of tainted pointer

Arithmetic operation performed on  pointer

 pointer dereferenced

Pointer from an unsecure source may be NULL or point to unknown memory

PRQA QA-C++
2810, 2811, 2812, 2813, 2814, 2820, 2821, 2822, 2823, 2824
PRQA QA-C

9.3

2810, 2811, 2812, 2813, 2814, 2820, 2821, 2822, 2823, 2824 

Fully implemented
PVS-Studio6.22V522, V595, V664, V713, V1004
SonarQube C/C++ Plugin

3.11

S2259
Splint

3.1.1



Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

Related Guidelines

Key here (explains table format and definitions)

CERT-CWE Mapping Notes

Key here for mapping notes

CWE-690 and EXP34-C

EXP34-C = Union( CWE-690, list) where list =

  • Dereferencing null pointers that were not returned by a function

CWE-252 and EXP34-C

Intersection( CWE-252, EXP34-C) = Ø

EXP34-C is a common consequence of ignoring function return values, but it is a distinct error, and can occur in other scenarios too.

Bibliography


#include <png.h> /* From libpng */ #include <string.h>   void func(png_structp png_ptr, int length, const void *user_data) { png_charp chunkdata; chunkdata = (png_charp)png_malloc(png_ptr, length + 1); /* ... */ memcpy(chunkdata, user_data, length); /* ... */  }
#include <png.h> /* From libpng */ #include <string.h>  void func(png_structp png_ptr, size_t length, const void *user_data) { png_charp chunkdata; if (length == SIZE_MAX) { /* Handle error */ }  chunkdata = (png_charp)png_malloc(png_ptr, length + 1); if (NULL == chunkdata) { /* Handle error */ } /* ... */ memcpy(chunkdata, user_data, length); /* ... */  }
#include <string.h> #include <stdlib.h>   void f(const char *input_str) { size_t size = strlen(input_str) + 1; char *c_str = (char *)malloc(size); memcpy(c_str, input_str, size); /* ... */ free(c_str); c_str = NULL; /* ... */ }
#include <string.h> #include <stdlib.h>   void f(const char *input_str) { size_t size; char *c_str;   if (NULL == input_str) { /* Handle error */ } size = strlen(input_str) + 1; c_str = (char *)malloc(size); if (NULL == c_str) { /* Handle error */ } memcpy(c_str, input_str, size); /* ... */ free(c_str); c_str = NULL; /* ... */ }
static unsigned int tun_chr_poll(struct file *file, poll_table *wait) { struct tun_file *tfile = file->private_data; struct tun_struct *tun = __tun_get(tfile); struct sock *sk = tun->sk; unsigned int mask = 0; if (!tun) return POLLERR; DBG(KERN_INFO "%s: tun_chr_poll\n", tun->dev->name); poll_wait(file, &tun->socket.wait, wait); if (!skb_queue_empty(&tun->readq)) mask |= POLLIN | POLLRDNORM; if (sock_writeable(sk) || (!test_and_set_bit(SOCK_ASYNC_NOSPACE, &sk->sk_socket->flags) && sock_writeable(sk))) mask |= POLLOUT | POLLWRNORM; if (tun->dev->reg_state != NETREG_REGISTERED) mask = POLLERR; tun_put(tun); return mask; }
static unsigned int tun_chr_poll(struct file *file, poll_table *wait) { struct tun_file *tfile = file->private_data; struct tun_struct *tun = __tun_get(tfile); struct sock *sk; unsigned int mask = 0; if (!tun) return POLLERR; sk = tun->sk; /* The remaining code is omitted because it is unchanged... */ }

Code performance always matters, and copying data is a common operation. Simple array copy code uses a loop to copy one value at a time. ISO C provides the and functions to do this efficiently, but are they faster, by how much, and under what conditions? This article benchmarks five common methods and four common compilers to find the fastest method to copy an array quickly.

How to copy an array

Let source data be an array of 4-byte integers to copy into a destination array. Both arrays are on the heap.

Method 1: Loop with array indexes

Use a loop with incrementing array indexes to copy from the source to the destination.

for ( size_t i = 0; i < length; ++i ) dst[i] = src[i];

Method 2: Loop with pointers

Use a loop with incrementing parallel source and destination pointers.

const int* s = src; int* d = dst; const int*const dend = dst + n; while ( d != dend ) *d++ = *s++;

Method 3: Loop with get/set function calls

Loop through the array and call and functions, such as accessor functions on a C++ data class, or from C++ array operator overloading.

for ( size_t i = 0; i < n; ++i ) set( dst, i, get( src, i ) );

The and functions used here are defined in the same file as:

int get( const int*const src, const size_t index ) { return src[index]; } int set( int*const dst, const size_t index, const int value ) { dst[index] = value; }

Method 4: Call memcpy( )

Call the ISO C function.

memcpy( (void*)dst, (void*)src, n * sizeof(int) );

Method 5: Call memmove( )

Call the ISO C function (same as but checks for source and destination overlap).

memmove( (void*)dst, (void*)src, n * sizeof(int) );

Untested methods

There are a lot of functions very similar to and , and usually implemented using them. Here are few:

  • Memory copy with void pointers:
    • POSIX's deprecated is identical to , but with swapped source and destination arguments.
    • GNU's returns a pointer to the byte after the last byte written, but is otherwise identical to .
    • POSIX's and Microsoft's stop the copy on a selected value, but are otherwise identical to .
    • Gnome's is a macro that usually maps to ).

    • Microsoft's and are identical to and .
    • Microsoft's and add a destination array length and do array bounds checking, but are otherwise identical to and
  • Memory copy with wide character pointers:
  • String copy with character pointers:
    • ISO C's and GNU's use character pointer arguments and stop the copy on a '\0' character, but are otherwise identical to and .
    • Microsoft's adds a destination array length and does array bounds checking, but is otherwise identical to .
  • String copy with wide character pointers:
    • ISO C's and GNU's use wide character pointer arguments and stop the copy on a '\0' wide character, but are otherwise identical to and .
    • Microsoft's adds a destination array length and does array bounds checking, but is otherwise identical to .

Test specification

The benchmarks reported below time the same code compiled with four common C/C++ compilers:

Compiler flags used here enable maximum code optimizations and 64-bit code tuned to a 2.6GHz Intel EMT64T Xeon E5-2670 "Sandy Bridge" processor. The process has a 32 Kbyte L1 cache, a 256 Kbyte L2 cache, and a 20 Mbyte L3 cache. The L1 cache supports two simultaneous CPU loads per clock cycle and an 8-byte bus for each load. For a 2.6GHz clock, the theoretical maximum bandwidth from the L1 cache to the CPU is (2.6GHz * 2 * 8) = 44 Gbytes/sec.

All tests were run on Linux using one core, and all tests were run multiple times and their results averaged.

The plots below show performance in Gbytes/sec (vertical axis) of each method as the array size (horizontal axis) is varied from 4 bytes to 64 Gbytes in powers of 2.

Benchmark results – compiled without optimizations

The plot below shows performance for code compiled with GCC without optimizations. CLANG/LLVM, ICC, and PGCC produce similar results. On the left the vertical axis runs from 0 to 40 Gbytes/sec. On the right, the plot has been zoomed in to the 0 to 2 Gbytes/sec range.

Compiler observations:

  • Unoptimized code performance for the looping methods is extremely poor at 1.5 Gbytes/sec or less compared to the theoretical maximum of 44 Gbytes/sec for the processor.

Method observations:

  • The loop methods are certainly faster than the get/set functions, but they're still slow compared to and , which use optimized LibC functions.

However, these results are for code without any compiler optimizations. Nobody should ever release unoptimized code.

Benchmark results – compiled with optimizations

The plots below show performance for code compiled using CLANG/LLVM, GCC, ICC, and PGCC using the maximum optimizations available on the test platform.

The mountain-shaped curve comes from a combination of loop overhead and the processor's cache speeds. For very small arrays, loop overhead dominates and performance is low. For mid-sized arrays, fast copying runs near the L1 cache speed and performance is high. As arrays get larger, they spill over from the L1 to L2 cache, L2 to L3, and L3 to memory, and performance drops. The curves smooth out L1/L2/L3 performance steps because the processor's prefetcher tries to stage data in the L1 cache ahead of need.

Compiler observations:

  • PGCC fails to optimize the loop with array indexes (blue) or get/set functions (yellow). The array index loop is a common code construct and should have been recognized and optimized, and the get/set functions should have been inlined. Additional, while the pointer loop (green) was optimized, its performance drops off suddenly at the 256 Kbyte array test, which is odd.
  • CLANG/LLVM fails to optimize the loop with pointers (green). This is a very common code construct and should have been optimized.

Method observations:

  • ICC is the undisputed star of the show with spectacular performance for everything except (purple), and even that is about as fast as the fastest results from the other compilers. The ICC code hits an amazing 42.5 Gbytes/sec, and just short of the 44 Gbyte/sec theoretical maximum. The get/set functions are clearly inlined and the looping methods have been optimized to as well as ICC's custom fast SSE .

Conclusions

  • Code from non-optimizing compilers is terrible and should never be released.
  • PGCC and CLANG/LLVM both missed optimizing common code constructs, producing unexpectedly bad performance.
  • is always the fastest method to copy data. A great compiler, like ICC, can optimize loops to do as well.

Further reading

Related articles at NadeauSoftware.com

Web articles

  • Please join me in welcoming to the SDL Rogues Gallery at Microsoft.com. This 2009 article briefly describes Microsoft's work to deprecate and functions in favor of . The thinking is that programmers forget how big the destination array is and accidentally copy beyond the end of it. Microsoft's brings the destination array size to programmer attention by adding it as a function argument, and letting the function check that the copy will fit. Detractors of Microsoft's stance (such as this one) point out that the same sloppy programming that creates a buffer overrun with works with if you use the wrong destination array size. So, creating a non-standard function accomplishes nothing except make it harder to port code to/from Windows. Here's Microsoft's full list of banned function calls. To date no other OSes have followed Microsoft's approach.

0 thoughts on “Pointer Assignment Vs Memcpy Linux

Leave a Reply

Your email address will not be published. Required fields are marked *