Is there a max array length limit in C++?

Is there a max array length limit in C++?

Nobody mentioned the limit on the size of the stack frame.

n

There are two places memory can be allocated:

n

    n

  • On the heap (dynamically allocated memory).
    nThe size limit here is a combination of available hardware and the OSs ability to simulate space by using other devices to temporarily store unused data (i.e. move pages to hard disk).
  • n

  • On the stack (Locally declared variables).
    nThe size limit here is compiler defined (with possible hardware limits). If you read the compiler documentation you can often tweak this size.
  • n

n

Thus if you allocate an array dynamically (the limit is large and described in detail by other posts.

n

int* a1 = new int[SIZE];  // SIZE limited only by OS/Hardwaren

n

Alternatively if the array is allocated on the stack then you are limited by the size of the stack frame. N.B. vectors and other containers have a small presence in the stack but usually the bulk of the data will be on the heap.

n

int a2[SIZE]; // SIZE limited by COMPILER to the size of the stack framen

There are two limits, both not enforced by C++ but rather by the hardware.

n

The first limit (should never be reached) is set by the restrictions of the size type used to describe an index in the array (and the size thereof). It is given by the maximum value the systems std::size_t can take. This data type is large enough to contain the size in bytes of any object

n

The other limit is a physical memory limit. The larger your objects in the array are, the sooner this limit is reached because memory is full. For example, a vector<int> of a given size n typically takes multiple times as much memory as an array of type vector<char> (minus a small constant value), since int is usually bigger than char. Therefore, a vector<char> may contain more items than a vector<int> before memory is full. The same counts for raw C-style arrays like int[] and char[].

n

Additionally, this upper limit may be influenced by the type of allocator used to construct the vector because an allocator is free to manage memory any way it wants. A very odd but nontheless conceivable allocator could pool memory in such a way that identical instances of an object share resources. This way, you could insert a lot of identical objects into a container that would otherwise use up all the available memory.

n

Apart from that, C++ doesnt enforce any limits.

Is there a max array length limit in C++?

Looking at it from a practical rather than theoretical standpoint, on a 32 bit Windows system, the maximum total amount of memory available for a single process is 2 GB. You can break the limit by going to a 64 bit operating system with much more physical memory, but whether to do this or look for alternatives depends very much on your intended users and their budgets. You can also extend it somewhat using PAE.

n

The type of the array is very important, as default structure alignment on many compilers is 8 bytes, which is very wasteful if memory usage is an issue. If you are using Visual C++ to target Windows, check out the #pragma pack directive as a way of overcoming this.

n

Another thing to do is look at what in memory compression techniques might help you, such as sparse matrices, on the fly compression, etc… Again this is highly application dependent. If you edit your post to give some more information as to what is actually in your arrays, you might get more useful answers.

n

Edit: Given a bit more information on your exact requirements, your storage needs appear to be between 7.6 GB and 76 GB uncompressed, which would require a rather expensive 64 bit box to store as an array in memory in C++. It raises the question why do you want to store the data in memory, where one presumes for speed of access, and to allow random access. The best way to store this data outside of an array is pretty much based on how you want to access it. If you need to access array members randomly, for most applications there tend to be ways of grouping clumps of data that tend to get accessed at the same time. For example, in large GIS and spatial databases, data often gets tiled by geographic area. In C++ programming terms you can override the [] array operator to fetch portions of your data from external storage as required.

Leave a Reply

Your email address will not be published.