Discussion:
pathlib.Path.is_file vs os.path.isfile difference
(too old to reply)
Albert-Jan Roskam
2024-03-08 18:03:40 UTC
Permalink
Hi,
I was replacing some os.path stuff with Pathlib and I discovered this:
Path(256 * "x").is_file() # OSError
os.path.isfile(256 * "x") # bool
Is this intended? Does pathlib try to resemble os.path as closely as
possible?
Best wishes,
Albert-Jan
Stefan Ram
2024-03-08 18:19:04 UTC
Permalink
Albert-Jan Roskam <***@hotmail.com> wrote or quoted:
>Path(256 * "x").is_file() # OSError
>os.path.isfile(256 * "x") # bool
>Is this intended?

When something is documented and behaves accordingly,
we assume it was intended.

>Does pathlib try to resemble os.path as closely as possible?

No.

The definition of "isfile" contains:

|except (OSError, ValueError):
| return False
Stefan Ram
2024-03-08 18:31:07 UTC
Permalink
Albert-Jan Roskam <***@hotmail.com> wrote or quoted:
>I was replacing some os.path stuff with Pathlib

Don't try to fix it if it ain't broke!
Lawrence D'Oliveiro
2024-03-08 20:47:51 UTC
Permalink
On 8 Mar 2024 18:31:07 GMT, Stefan Ram wrote:

> Albert-Jan Roskam <***@hotmail.com> wrote or quoted:
>
>>I was replacing some os.path stuff with Pathlib
>
> Don't try to fix it if it ain't broke!

I can’t see the point of pathlib. It just seems to be trying to paper over
the differences between Windows and *nix systems. Does anybody still care
about being compatible with native-Windows any more? After all, that’s
what WSL is for.
Thomas Passin
2024-03-08 18:35:47 UTC
Permalink
On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
> Hi,
> I was replacing some os.path stuff with Pathlib and I discovered this:
> Path(256 * "x").is_file() # OSError
> os.path.isfile(256 * "x") # bool
> Is this intended? Does pathlib try to resemble os.path as closely as
> possible?

You must have an very old version of Python. I'm running 3.12.2 and it
returns False. Either that or that path name exists and throws some
kind of unexpected exception.

The Python docs say

"Return True if the path points to a regular file (or a symbolic link
pointing to a regular file), False if it points to another kind of file.

False is also returned if the path doesn’t exist or is a broken symlink;
other errors (such as permission errors) are propagated"
Grant Edwards
2024-03-08 19:21:39 UTC
Permalink
On 2024-03-08, Thomas Passin via Python-list <python-***@python.org> wrote:
> On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
>> Hi,
>> I was replacing some os.path stuff with Pathlib and I discovered this:
>> Path(256 * "x").is_file() # OSError
>> os.path.isfile(256 * "x") # bool
>> Is this intended? Does pathlib try to resemble os.path as closely as
>> possible?
>
> You must have an very old version of Python. I'm running 3.12.2 and it
> returns False.

It throws OSError with Python 3.11.8 on Linux.

$ python
Python 3.11.8 (main, Feb 23 2024, 16:11:29) [GCC 13.2.1 20240113] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> pathlib.Path(256 * "x").is_file()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.11/pathlib.py", line 1267, in is_file
return S_ISREG(self.stat().st_mode)
^^^^^^^^^^^
File "/usr/lib/python3.11/pathlib.py", line 1013, in stat
return os.stat(self, follow_symlinks=follow_symlinks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 36] File name too long: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
>>>
>>> import os
>>> os.path.isfile(256 * "x")
False
Grant Edwards
2024-03-08 19:30:51 UTC
Permalink
On 2024-03-08, Grant Edwards via Python-list <python-***@python.org> wrote:
> On 2024-03-08, Thomas Passin via Python-list <python-***@python.org> wrote:
>> On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
>>> Hi,
>>> I was replacing some os.path stuff with Pathlib and I discovered this:
>>> Path(256 * "x").is_file() # OSError
>>> os.path.isfile(256 * "x") # bool
>>> Is this intended? Does pathlib try to resemble os.path as closely as
>>> possible?
>>
>> You must have an very old version of Python. I'm running 3.12.2 and it
>> returns False.
>
> It throws OSError with Python 3.11.8 on Linux.

> OSError: [Errno 36] File name too long: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

On all of the Linux filesystems I know about, the max length for a
filename is 255 bytes, so the OSError is too surprising, and it does
seem to follow the documentation.

>>>> import os
>>>> os.path.isfile(256 * "x")
> False

However, os.path.isfile() apprently masks that error somehow and
returns False instead.

I notice that the os.path.isfile() documentation does not specify what
happens if the path is not a file or is illegal. It only specifies
that True is returned if the path is a regular file. Presumably
something other than "return True" is supposed to happen, but exactly
what is not specified.
Grant Edwards
2024-03-08 19:37:24 UTC
Permalink
On 2024-03-08, Grant Edwards via Python-list <python-***@python.org> wrote:

>> OSError: [Errno 36] File name too long: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
>
> On all of the Linux filesystems I know about, the max length for a
> filename is 255 bytes, so the OSError is too surprising, and it does
> seem to follow the documentation.

Doh. I meant "is not too surprising".
Thomas Passin
2024-03-08 20:06:24 UTC
Permalink
On 3/8/2024 2:21 PM, Grant Edwards via Python-list wrote:
> On 2024-03-08, Thomas Passin via Python-list <python-***@python.org> wrote:
>> On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
>>> Hi,
>>> I was replacing some os.path stuff with Pathlib and I discovered this:
>>> Path(256 * "x").is_file() # OSError
>>> os.path.isfile(256 * "x") # bool
>>> Is this intended? Does pathlib try to resemble os.path as closely as
>>> possible?
>>
>> You must have an very old version of Python. I'm running 3.12.2 and it
>> returns False.
>
> It throws OSError with Python 3.11.8 on Linux.

Sorry, I should have said on Windows.

>
> $ python
> Python 3.11.8 (main, Feb 23 2024, 16:11:29) [GCC 13.2.1 20240113] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import pathlib
>>>> pathlib.Path(256 * "x").is_file()
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/lib/python3.11/pathlib.py", line 1267, in is_file
> return S_ISREG(self.stat().st_mode)
> ^^^^^^^^^^^
> File "/usr/lib/python3.11/pathlib.py", line 1013, in stat
> return os.stat(self, follow_symlinks=follow_symlinks)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> OSError: [Errno 36] File name too long: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
>>>>
>>>> import os
>>>> os.path.isfile(256 * "x")
> False
>
Albert-Jan Roskam
2024-03-08 22:14:16 UTC
Permalink
On Mar 8, 2024 19:35, Thomas Passin via Python-list
<python-***@python.org> wrote:

On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
> Hi,
> I was replacing some os.path stuff with Pathlib and I discovered
this:
> Path(256 * "x").is_file() # OSError
> os.path.isfile(256 * "x") # bool
> Is this intended? Does pathlib try to resemble os.path as closely
as
> possible?

You must have an very old version of Python. I'm running 3.12.2 and it
returns False. Either that or that path name exists and throws some
kind of unexpected exception.

====
Hi, I tested this with Python 3.8. Good to know that this was fixed!
Thomas Passin
2024-03-08 22:57:57 UTC
Permalink
On 3/8/2024 5:14 PM, Albert-Jan Roskam wrote:
>
>
> On Mar 8, 2024 19:35, Thomas Passin via Python-list
> <python-***@python.org> wrote:
>
> On 3/8/2024 1:03 PM, Albert-Jan Roskam via Python-list wrote:
> >     Hi,
> >     I was replacing some os.path stuff with Pathlib and I
> discovered this:
> >     Path(256 * "x").is_file()  # OSError
> >     os.path.isfile(256 * "x")  # bool
> >     Is this intended? Does pathlib try to resemble os.path as
> closely as
> >     possible?
>
> You must have an very old version of Python.  I'm running 3.12.2 and it
> returns False.  Either that or that path name exists and throws some
> kind of unexpected exception.
>
>
>
> ====
>
> Hi, I tested this with Python 3.8. Good to know that this was fixed!

We just learned a few posts back that it might be specific to Linux; I
ran it on Windows.
Grant Edwards
2024-03-08 23:39:51 UTC
Permalink
On 2024-03-08, Thomas Passin via Python-list <python-***@python.org> wrote:
>
>> Hi, I tested this with Python 3.8. Good to know that this was fixed!
>
> We just learned a few posts back that it might be specific to Linux; I
> ran it on Windows.

On Linux, the limit is imposed by the filesystem. Most of the "real"
filesystems on Linux have a 255 character limit, a few support 256,
and some of the legacy filesystems have lower limits. Reiser4 is the
only one that's even remotely common which supports more than 256 --
according to https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
it supports filenames up to 3976 bytes long.

NB: The behavior when the limit is exceeded might also vary from one
filesystem to another.

In any case, the pathlib docs for is_file() are explicit: any errors
from the underlying OS and libraries will be propogated. There is
nothing to fix.

https://docs.python.org/3/library/pathlib.html#pathlib.Path.is_file

Path.is_file()

Return True if the path points to a regular file (or a symbolic
link pointing to a regular file), False if it points to another
kind of file.

False is also returned if the path doesn’t exist or is a broken
symlink; other errors (such as permission errors) are propagated.
Barry
2024-03-10 10:17:45 UTC
Permalink
> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list <python-***@python.org> wrote:
>
> We just learned a few posts back that it might be specific to Linux; I ran it on Windows.

Depending on the exact win32 api used there is a 257 limit on windows.
The 257 includes 2 for the device, C:, and 255 for the path part that will use 1 for the leading \. Getting an error for a name that is 255 is not surprising.

Other api allow for 65535 limit, not sure on its additional limits.

Barry
Thomas Passin
2024-03-10 11:59:44 UTC
Permalink
On 3/10/2024 6:17 AM, Barry wrote:
>
>
>> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list <python-***@python.org> wrote:
>>
>> We just learned a few posts back that it might be specific to Linux; I ran it on Windows.
>
> Depending on the exact win32 api used there is a 257 limit on windows.
> The 257 includes 2 for the device, C:, and 255 for the path part that will use 1 for the leading \. Getting an error for a name that is 255 is not surprising.
>
> Other api allow for 65535 limit, not sure on its additional limits.

I seem to remember there is a setting to allow longer paths, but I
forget any details.
Albert-Jan Roskam
2024-03-10 13:33:02 UTC
Permalink
On Mar 10, 2024 12:59, Thomas Passin via Python-list
<python-***@python.org> wrote:

On 3/10/2024 6:17 AM, Barry wrote:
>
>
>> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list
<python-***@python.org> wrote:
>>
>> We just learned a few posts back that it might be specific to Linux;
I ran it on Windows.
>
> Depending on the exact win32 api used there is a 257 limit on windows.
> The 257 includes 2 for the device, C:, and 255 for the path part that
will use 1 for the leading \. Getting an error for a name that is 255 is
not surprising.
>
> Other api allow for 65535 limit, not sure on its additional limits.

I seem to remember there is a setting to allow longer paths, but I
forget any details.

=====
You mean the "\\?\" prefix?
https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry
Thomas Passin
2024-03-10 14:09:53 UTC
Permalink
On 3/10/2024 9:33 AM, Albert-Jan Roskam wrote:
>
>
> On Mar 10, 2024 12:59, Thomas Passin via Python-list
> <python-***@python.org> wrote:
>
> On 3/10/2024 6:17 AM, Barry wrote:
> >
> >
> >> On 8 Mar 2024, at 23:19, Thomas Passin via Python-list
> <python-***@python.org> wrote:
> >>
> >> We just learned a few posts back that it might be specific to
> Linux; I ran it on Windows.
> >
> > Depending on the exact win32 api used there is a 257 limit on
> windows.
> > The 257 includes 2 for the device, C:, and 255 for the path part
> that will use 1 for the leading \. Getting an error for a name that
> is 255 is not surprising.
> >
> > Other api allow for 65535 limit, not sure on its additional limits.
>
> I seem to remember there is a setting to allow longer paths, but I
> forget any details.
>
>
>
> =====
>
> You mean the "\\?\" prefix?
>
> https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry

That and there's a registry setting:

https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation
Barry
2024-03-10 22:09:05 UTC
Permalink
> On 10 Mar 2024, at 14:49, Thomas Passin via Python-list <python-***@python.org> wrote:
>
> That and there's a registry setting:
>
> https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation
>
Yep that and rules about size of parts of the path.

Barry
Loading...