Bug 60191

Summary: libapr computes incorrect size for dirent structure
Product: APR Reporter: Alexander Pyhalov <apyhalov>
Component: APRAssignee: Apache Portable Runtime bugs mailinglist <bugs>
Status: NEW ---    
Severity: normal    
Priority: P2    
Version: 1.5.2   
Target Milestone: ---   
Hardware: PC   
OS: Solaris   

Description Alexander Pyhalov 2016-09-30 09:35:42 UTC
This is an upstream bug for https://www.illumos.org/issues/7394 . 

In apr_dir_open we have the following code: 

    /* On some platforms (e.g., Linux+GNU libc), d_name[] in struct 
     * dirent is declared with enough storage for the name.  On other
     * platforms (e.g., Solaris 8 for Intel), d_name is declared as a
     * one-byte array.  Note: gcc evaluates this at compile time.
     */
    apr_size_t dirent_size = 
        sizeof(*(*new)->entry) +
        (sizeof((*new)->entry->d_name) > 1 ? 0 : 255);


Which works incorrectly in both cases. On illumos SPARC this just allocates 0 bytes to directory names, on Intel (where d_name is char[1] - 255 bytes).
It should just use PATH_MAX+1 (on illumos PATH_MAX is 1024).
Something like this :

--- file_io/unix/dir.c.1        2016-09-30 11:54:20.721126026 +0300
+++ file_io/unix/dir.c  2016-09-30 12:01:08.345539555 +0300
@@ -78,7 +78,7 @@
      */
     apr_size_t dirent_size = 
         sizeof(*(*new)->entry) +
-        (sizeof((*new)->entry->d_name) > 1 ? 0 : 255);
+        PATH_MAX+1;
     DIR *dir = opendir(dirname);
 
     if (!dir) {
Comment 1 Alexander Pyhalov 2016-09-30 10:59:26 UTC
Sorry, path solves the problem, but is incorrect.

sizeof(struct dirent) includes d_name array in the result,
it already has an extra space (minimum one byte) for terminating '\0',
but the second part of this expression is incorrect if d_name array
has size more than one byte. On SPARC is has 3 bytes and this part
of the expression returns 0, so, we have for names only 3 bytes
including '\0' which is incorrect.
Comment 2 William A. Rowe Jr. 2016-09-30 20:35:54 UTC
Why not

-        (sizeof((*new)->entry->d_name) > 1 ? 0 : 255);
+        PATH_MAX+1 - sizeof((*new)->entry->d_name);

to allocate a sufficient d_name space while deducting any redundant bytes 
from the d_name elt?
Comment 3 Yann Ylavic 2016-09-30 21:06:07 UTC
(In reply to William A. Rowe Jr. from comment #2)
> Why not
> 
> -        (sizeof((*new)->entry->d_name) > 1 ? 0 : 255);
> +        PATH_MAX+1 - sizeof((*new)->entry->d_name);

Or maybe:

+    apr_size_t dname_size = sizeof((*new)->entry->d_name);
     apr_size_t dirent_size = 
         sizeof(*(*new)->entry) +
-        (sizeof((*new)->entry->d_name) > 1 ? 0 : 255);
+        (d_name_size > PATH_MAX ? dname_size : PATH_MAX + 1) -
+        dname_size;

to preserve sizeof(d_name) if it's above PATH_MAX already.
Comment 4 Ruediger Pluem 2016-10-04 18:36:28 UTC
(In reply to Yann Ylavic from comment #3)
> (In reply to William A. Rowe Jr. from comment #2)
> > Why not
> > 
> > -        (sizeof((*new)->entry->d_name) > 1 ? 0 : 255);
> > +        PATH_MAX+1 - sizeof((*new)->entry->d_name);
> 
> Or maybe:
> 
> +    apr_size_t dname_size = sizeof((*new)->entry->d_name);
>      apr_size_t dirent_size = 
>          sizeof(*(*new)->entry) +
> -        (sizeof((*new)->entry->d_name) > 1 ? 0 : 255);
> +        (d_name_size > PATH_MAX ? dname_size : PATH_MAX + 1) -
> +        dname_size;
> 
> to preserve sizeof(d_name) if it's above PATH_MAX already.

As far as I understand the current calculation and the one Bill proposed can be done at compile time while yours need to be done at runtime. Probably using a #define dname_size could fix this.
Comment 5 Yann Ylavic 2016-10-04 21:20:50 UTC
Since dname_size is constant it can/will be optimized out at compile time too.
The issue is that my version (and Bill's) over allocate memory by using PATH_MAX instead NAME_MAX, so we could possibly go for something like:

Index: file_io/unix/dir.c
===================================================================
--- file_io/unix/dir.c	(revision 1762122)
+++ file_io/unix/dir.c	(working copy)
@@ -76,9 +76,15 @@ apr_status_t apr_dir_open(apr_dir_t **new, const c
      * platforms (e.g., Solaris 8 for Intel), d_name is declared as a
      * one-byte array.  Note: gcc evaluates this at compile time.
      */
+#if defined(NAME_MAX) && NAME_MAX > 255
+#define DIRENT_NAME_MAX NAME_MAX
+#else
+#define DIRENT_NAME_MAX 255
+#endif
     apr_size_t dirent_size = 
         sizeof(*(*new)->entry) +
-        (sizeof((*new)->entry->d_name) > 1 ? 0 : 255);
+        (sizeof((*new)->entry->d_name) > DIRENT_NAME_MAX ? 0
+         : DIRENT_NAME_MAX + 1 - sizeof((*new)->entry->d_name));
     DIR *dir = opendir(dirname);
 
     if (!dir) {
_


The best we could do would be to use fpathconf() like proposed in the dirent_buf_size() function from https://womble.decadent.org.uk/readdir_r-advisory.html (there may be some platforms supporting file names longer than NAME_MAX, but with a lower sizeof(d_name)...).
This is not compile time though, but we are supposedly not in a fast path when using opendir()/readdir() :)