Extract lines from file, add to new files

[Rich Shepard]

It's been several years since I've needed to write a python script so I'm
asking for advice to get me started with a brief script to separate names
and email addresses in one file into two separate files: salutation.txt and
emails.txt.

An example of the input file:

Calvin
***@example.com

Hobbs
***@some.com

Nancy
***@herown.com

Sluggo
***@another.com

Having extracted salutations and addresses I'll write a bash script using
sed and mailx to associate a message file with each name and email address.

I'm unsure where to start given my lack of recent experience.

TIA,

Rich

[Piergiorgio Sartor]

On 11/01/2024 19.08, Rich Shepard wrote:
> It's been several years since I've needed to write a python script so I'm
> asking for advice to get me started with a brief script to separate names
> and email addresses in one file into two separate files: salutation.txt and
> emails.txt.
>
> An example of the input file:
>
> Calvin
> ***@example.com
>
> Hobbs
> ***@some.com
>
> Nancy
> ***@herown.com
>
> Sluggo
> ***@another.com
>
> Having extracted salutations and addresses I'll write a bash script using
> sed and mailx to associate a message file with each name and email address.

Why not to use bash script for all?

bye,

--

piergiorgio

[Rich Shepard]

On Thu, 11 Jan 2024, Piergiorgio Sartor via Python-list wrote:

> Why not to use bash script for all?

Piergiorgio,

That's certainly a possibility, and may well be better than python for this
task.

Thank you,

Rich

[Mirko]

Am 11.01.24 um 20:53 schrieb Rich Shepard via Python-list:
> On Thu, 11 Jan 2024, Piergiorgio Sartor via Python-list wrote:
>
>> Why not to use bash script for all?
>
> Piergiorgio,
>
> That's certainly a possibility, and may well be better than python
> for this
> task.
>
> Thank you,
>
> Rich

awk '/@/ {print >>"emails.txt";next};NF{print >>"salutation.txt"}'
input.txt


SCNR ;-)

[MRAB]

On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
> It's been several years since I've needed to write a python script so I'm
> asking for advice to get me started with a brief script to separate names
> and email addresses in one file into two separate files: salutation.txt and
> emails.txt.
>
> An example of the input file:
>
> Calvin
> ***@example.com
>
> Hobbs
> ***@some.com
>
> Nancy
> ***@herown.com
>
> Sluggo
> ***@another.com
>
> Having extracted salutations and addresses I'll write a bash script using
> sed and mailx to associate a message file with each name and email address.
>
> I'm unsure where to start given my lack of recent experience.
>
From the look of it:

1. If the line is empty, ignore it.

2. If the line contains "@", it's an email address.

3. Otherwise, it's a name.

[Stefan Ram]

Rich Shepard <***@appl-ecosys.com> writes:
>separate names and email addresses

(Warning: execution of the following script will overwrite
[DELETE!] files opened with 'w'!)

# create source
data = '''Calvin\***@example.com\nHobbs\***@some.com
Nancy\***@herown.com\nSluggo\***@another.com'''
fn = "data20240111192219+0100"
with open( fn, "w" )as sink: print( data, file=sink )

# split source into two files
with open( fn )as source: file = source.read()
lines = file.split( '\n' )[ :-1 ]
with open( 'salutation.txt', 'w' )as f: print( lines[ 0::2 ], file=f )
with open( 'emails.txt', 'w' )as f: print( lines[ 1::2 ], file=f )

# show the two result files
with open( 'salutation.txt' )as file: print( file.read() )
with open( 'emails.txt' )as file: print( file.read() )

[Rich Shepard]

On Thu, 11 Jan 2024, MRAB via Python-list wrote:

> From the look of it:
> 1. If the line is empty, ignore it.
> 2. If the line contains "@", it's an email address.
> 3. Otherwise, it's a name.

MRAB,

Thanks. I'll take it from here.

Regards,

Rich

[Mats Wichmann]

On 1/11/24 11:27, MRAB via Python-list wrote:
> On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
>> It's been several years since I've needed to write a python script so I'm
>> asking for advice to get me started with a brief script to separate names
>> and email addresses in one file into two separate files:
>> salutation.txt and
>> emails.txt.
>>
>> An example of the input file:
>>
>> Calvin
>> ***@example.com
>>
>> Hobbs
>> ***@some.com
>>
>> Nancy
>> ***@herown.com
>>
>> Sluggo
>> ***@another.com
>>
>> Having extracted salutations and addresses I'll write a bash script using
>> sed and mailx to associate a message file with each name and email
>> address.
>>
>> I'm unsure where to start given my lack of recent experience.
>>
> From the look of it:
>
> 1. If the line is empty, ignore it.
>
> 2. If the line contains "@", it's an email address.
>
> 3. Otherwise, it's a name.
>

4. Don't assume it's going to be "plain text" if the email info is
harvested from external sources (like incoming emails) - you'll end up
stumbling over a 誰かのユーザー from somewhere. Process as bytes, or be really
careful about which encodings you allow - which for email "names" is
something you can't actually control.

[Rich Shepard]

On Thu, 11 Jan 2024, Mats Wichmann via Python-list wrote:

> 4. Don't assume it's going to be "plain text" if the email info is
> harvested from external sources (like incoming emails) - you'll end up
> stumbling over a $BC/$+$N%f!<%6!<(J from somewhere. Process as bytes, or be
> really careful about which encodings you allow - which for email "names"
> is something you can't actually control.

Mats,

Not an issue for me.

Regards,

Rich

[Thomas Passin]

On 1/11/2024 1:27 PM, MRAB via Python-list wrote:
> On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
>> It's been several years since I've needed to write a python script so I'm
>> asking for advice to get me started with a brief script to separate names
>> and email addresses in one file into two separate files:
>> salutation.txt and
>> emails.txt.
>>
>> An example of the input file:
>>
>> Calvin
>> ***@example.com
>>
>> Hobbs
>> ***@some.com
>>
>> Nancy
>> ***@herown.com
>>
>> Sluggo
>> ***@another.com
>>
>> Having extracted salutations and addresses I'll write a bash script using
>> sed and mailx to associate a message file with each name and email
>> address.
>>
>> I'm unsure where to start given my lack of recent experience.
>>
> From the look of it:
>
> 1. If the line is empty, ignore it.
>
> 2. If the line contains "@", it's an email address.
>
> 3. Otherwise, it's a name.

You could think about a single Python script that looks through your
input file and constructs all the message files without ever writing
separate salutation and address files at all. Then you wouldn't need to
write the sed and mailx scripts. It shouldn't be much harder than
peeling out the names and addresses into separate files.

If you haven't written any Python for some years, the preferred way to
read and write files is using a "with" statement, like this:

with open('email_file.txt', encoding = 'utf-8') as f:
lines = f.readlines()
for line in lines:
if not line.strip(): # Skip blank lines
continue
# Do something with this line

You don't need to close the file because when the "with" block ends the
file will be closed for you.

If the encoding is not utf-8 and you know what it will be, use that
encoding instead.

[Left Right]

By the way, in an attempt to golf this problem, I discovered this,
which seems like a parser problem:

This is what Python tells me about its grammar:

with_stmt:
| 'with' '(' ','.with_item+ ','? ')' ':' block
| 'with' ','.with_item+ ':' [TYPE_COMMENT] block
| ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block
| ASYNC 'with' ','.with_item+ ':' [TYPE_COMMENT] block

with_item:
| expression 'as' star_target &(',' | ')' | ':')
| expression

>From which I figured why not something like this:

with (open('example.txt', 'r'), open('emails.txt', 'w'),
open('salutations.txt', 'w')) as e, m, s:
for line in e:
if line.strip():
(m if '@' in line else s).write(line)

Which, surprise, parsers! But it seems like it's parse is wrong,
because running this I get:

❯ python ./split_emails.py
Traceback (most recent call last):
File "/home/?/doodles/python/./split_emails.py", line 1, in <module>
with (open('example.txt', 'r'), open('emails.txt', 'w'),
open('salutations.txt', 'w')) as e, m, s:
TypeError: 'tuple' object does not support the context manager protocol

It seems to me it shouldn't have been parsed as a tuple. The
parenthesis should've been interpreted just as a decoration.

NB. I'm using 3.11.6.

On Thu, Jan 11, 2024 at 10:20 PM Thomas Passin via Python-list
<python-***@python.org> wrote:
>
> On 1/11/2024 1:27 PM, MRAB via Python-list wrote:
> > On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
> >> It's been several years since I've needed to write a python script so I'm
> >> asking for advice to get me started with a brief script to separate names
> >> and email addresses in one file into two separate files:
> >> salutation.txt and
> >> emails.txt.
> >>
> >> An example of the input file:
> >>
> >> Calvin
> >> ***@example.com
> >>
> >> Hobbs
> >> ***@some.com
> >>
> >> Nancy
> >> ***@herown.com
> >>
> >> Sluggo
> >> ***@another.com
> >>
> >> Having extracted salutations and addresses I'll write a bash script using
> >> sed and mailx to associate a message file with each name and email
> >> address.
> >>
> >> I'm unsure where to start given my lack of recent experience.
> >>
> > From the look of it:
> >
> > 1. If the line is empty, ignore it.
> >
> > 2. If the line contains "@", it's an email address.
> >
> > 3. Otherwise, it's a name.
>
> You could think about a single Python script that looks through your
> input file and constructs all the message files without ever writing
> separate salutation and address files at all. Then you wouldn't need to
> write the sed and mailx scripts. It shouldn't be much harder than
> peeling out the names and addresses into separate files.
>
> If you haven't written any Python for some years, the preferred way to
> read and write files is using a "with" statement, like this:
>
> with open('email_file.txt', encoding = 'utf-8') as f:
> lines = f.readlines()
> for line in lines:
> if not line.strip(): # Skip blank lines
> continue
> # Do something with this line
>
> You don't need to close the file because when the "with" block ends the
> file will be closed for you.
>
> If the encoding is not utf-8 and you know what it will be, use that
> encoding instead.
>
> --
> https://mail.python.org/mailman/listinfo/python-list

[Left Right]

Ah, nevermind. I need to be more careful, there isn't an "'as'
star_target" after the first rule.

On Thu, Jan 11, 2024 at 10:33 PM Left Right <***@gmail.com> wrote:
>
> By the way, in an attempt to golf this problem, I discovered this,
> which seems like a parser problem:
>
> This is what Python tells me about its grammar:
>
> with_stmt:
> | 'with' '(' ','.with_item+ ','? ')' ':' block
> | 'with' ','.with_item+ ':' [TYPE_COMMENT] block
> | ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block
> | ASYNC 'with' ','.with_item+ ':' [TYPE_COMMENT] block
>
> with_item:
> | expression 'as' star_target &(',' | ')' | ':')
> | expression
>
> From which I figured why not something like this:
>
> with (open('example.txt', 'r'), open('emails.txt', 'w'),
> open('salutations.txt', 'w')) as e, m, s:
> for line in e:
> if line.strip():
> (m if '@' in line else s).write(line)
>
> Which, surprise, parsers! But it seems like it's parse is wrong,
> because running this I get:
>
> ❯ python ./split_emails.py
> Traceback (most recent call last):
> File "/home/?/doodles/python/./split_emails.py", line 1, in <module>
> with (open('example.txt', 'r'), open('emails.txt', 'w'),
> open('salutations.txt', 'w')) as e, m, s:
> TypeError: 'tuple' object does not support the context manager protocol
>
> It seems to me it shouldn't have been parsed as a tuple. The
> parenthesis should've been interpreted just as a decoration.
>
> NB. I'm using 3.11.6.
>
> On Thu, Jan 11, 2024 at 10:20 PM Thomas Passin via Python-list
> <python-***@python.org> wrote:
> >
> > On 1/11/2024 1:27 PM, MRAB via Python-list wrote:
> > > On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
> > >> It's been several years since I've needed to write a python script so I'm
> > >> asking for advice to get me started with a brief script to separate names
> > >> and email addresses in one file into two separate files:
> > >> salutation.txt and
> > >> emails.txt.
> > >>
> > >> An example of the input file:
> > >>
> > >> Calvin
> > >> ***@example.com
> > >>
> > >> Hobbs
> > >> ***@some.com
> > >>
> > >> Nancy
> > >> ***@herown.com
> > >>
> > >> Sluggo
> > >> ***@another.com
> > >>
> > >> Having extracted salutations and addresses I'll write a bash script using
> > >> sed and mailx to associate a message file with each name and email
> > >> address.
> > >>
> > >> I'm unsure where to start given my lack of recent experience.
> > >>
> > > From the look of it:
> > >
> > > 1. If the line is empty, ignore it.
> > >
> > > 2. If the line contains "@", it's an email address.
> > >
> > > 3. Otherwise, it's a name.
> >
> > You could think about a single Python script that looks through your
> > input file and constructs all the message files without ever writing
> > separate salutation and address files at all. Then you wouldn't need to
> > write the sed and mailx scripts. It shouldn't be much harder than
> > peeling out the names and addresses into separate files.
> >
> > If you haven't written any Python for some years, the preferred way to
> > read and write files is using a "with" statement, like this:
> >
> > with open('email_file.txt', encoding = 'utf-8') as f:
> > lines = f.readlines()
> > for line in lines:
> > if not line.strip(): # Skip blank lines
> > continue
> > # Do something with this line
> >
> > You don't need to close the file because when the "with" block ends the
> > file will be closed for you.
> >
> > If the encoding is not utf-8 and you know what it will be, use that
> > encoding instead.
> >
> > --
> > https://mail.python.org/mailman/listinfo/python-list

[dn]

On 12/01/24 10:33, Left Right via Python-list wrote:
> By the way, in an attempt to golf this problem, I discovered this,
> which seems like a parser problem:
>
> This is what Python tells me about its grammar:
>
> with_stmt:
> | 'with' '(' ','.with_item+ ','? ')' ':' block
> | 'with' ','.with_item+ ':' [TYPE_COMMENT] block
> | ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block
> | ASYNC 'with' ','.with_item+ ':' [TYPE_COMMENT] block
>
> with_item:
> | expression 'as' star_target &(',' | ')' | ':')
> | expression
>
> From which I figured why not something like this:
>
> with (open('example.txt', 'r'), open('emails.txt', 'w'),
> open('salutations.txt', 'w')) as e, m, s:
> for line in e:
> if line.strip():
> (m if '@' in line else s).write(line)
>
> Which, surprise, parsers! But it seems like it's parse is wrong,
> because running this I get:
>
> ❯ python ./split_emails.py
> Traceback (most recent call last):
> File "/home/?/doodles/python/./split_emails.py", line 1, in <module>
> with (open('example.txt', 'r'), open('emails.txt', 'w'),
> open('salutations.txt', 'w')) as e, m, s:
> TypeError: 'tuple' object does not support the context manager protocol
>
> It seems to me it shouldn't have been parsed as a tuple. The
> parenthesis should've been interpreted just as a decoration.
>
> NB. I'm using 3.11.6.
A series of comma-separated items will be parsed as a tuple (some people
think it is bounding-parentheses which define).

In this case, the issue is 'connecting' the context-manager "expression"
with its (as) "target". These should be more-closely paired:-

with ( open( 'example.txt', 'r', ) as e,
open( 'emails.txt', 'w', ) as m,
open( 'salutations.txt', 'w', ) as s
):

(NB code not executed here)


A data-architecture of having related-data in separated serial-files is
NOT recommendable!

--
Regards,
=dn

[Chris Angelico]

On Fri, 12 Jan 2024 at 08:56, Left Right via Python-list
<python-***@python.org> wrote:
>
> By the way, in an attempt to golf this problem, I discovered this,
> which seems like a parser problem:

When you jump immediately to "this is a bug", all you do is make
yourself look like an idiot. Unsurprisingly, this is NOT a bug, this
is simply that you didn't understand what was going on. The grammar
isn't easy to read, and it's usually better to read the documentation
instead.

(Plus, golfing isn't really a goal in Python, and you didn't shorten
the code by much at all. Good job.)

ChrisA

[dn]

On 12/01/24 12:56, Chris Angelico via Python-list wrote:
> On Fri, 12 Jan 2024 at 08:56, Left Right via Python-list
> <python-***@python.org> wrote:
>>
>> By the way, in an attempt to golf this problem, I discovered this,
>> which seems like a parser problem:
>
> When you jump immediately to "this is a bug", all you do is make

"seems"!

but yes, it is a (much) less-likely explanation.


> yourself look like an idiot. Unsurprisingly, this is NOT a bug, this
> is simply that you didn't understand what was going on. The grammar
> isn't easy to read, and it's usually better to read the documentation
> instead.

Those of us who studied Computer Science may well have been
taught/expected to learn how to read [modified] BNF - indeed to have
worked in that (cf coding in Python).

Accordingly, the English text is likely easier to understand, but
sometimes the BNF offers finer-detail or can be used to clarify some
mis- or insufficiently-understood aspect of the text. IMHO/YMMV/etc...


> (Plus, golfing isn't really a goal in Python, and you didn't shorten
> the code by much at all. Good job.)

I took my hat off to the poster, being prepared to dive-in and do this.
Accordingly, was more than happy to help set him/her back onto 'the
straight and narrow'.

(yes it was a BNF-failing - which, credit where credit's due, I think
was realised at the same time as response was typed)

How many others just want us to do all their thinking for them?
(there's a rude comment about wiping noses - but probably a step too far
wrt the CoC)

--
Regards,
=dn

[Grizzy Adams]

Thursday, January 11, 2024 at 10:44, Rich Shepard via Python-list wrote:
Re: Extract lines from file, add to (at least in part)

>On Thu, 11 Jan 2024, MRAB via Python-list wrote:

>> From the look of it:
>> 1. If the line is empty, ignore it.
>> 2. If the line contains "@", it's an email address.
>> 3. Otherwise, it's a name.

If that is it all? a simple Grep would do (and save on the blank line)

[a***@gmail.com]

If the data in the input file is exactly as described and consists of
alternating lines containing a name and email address, or perhaps an
optional blank line, then many solutions are possible using many tools
including python programs.

But is the solution a good one for some purpose? The two output files may
end up being out of sync for all kinds of reasons. One of many "errors" can
happen if multiple lines in a row do not have an "@" or a person's name
does, for example. What if someone supplied more than one email address with
a comma separator? This may not be expected but could cause problems.

Some of the other tools mentioned would not care and produce garbage. Grep
as an example could be run twice asking for lines with an "@" and then lines
without. In this case, that would be trivial. Blank lines, or ones with just
whitespace, might need another pass to be omitted.

But a real challenge would be to parse the file in a language like Python
and find all VALID stretches in the data and construct a data structure
containing either a valid name or something specific like "ANONYMOUS"
alongside an email address. These may be written out as soon as it is
considered valid, or collected in something like a list. You can do further
processing if you want the results in some order or remove duplicates or bad
email addresses and so on. In that scenario, the two files would be written
out at the end.

Python can do the above while some of the other tools mentioned are not
really designed for it. Further, many of the tools are not generally
available everywhere.

Another question is why it makes sense to produce two output files to
contain the data that may not be linked and would not be easy to edit and
keep synchronized such as to remove or add entries. There are many ways to
save the data that might be more robust for many purposes. It looks like the
application intended is a sort of form letter merge where individual emails
will be sent that contain a personalized greeting. Unless that application
has already been written, there are many other ways that make sense. One
obvious one is to save the data in a databases as columns in a table. Other
ones are to write one file with entries easily parsed out such as:

NAME: name | EMAIL: email

Whatever the exact design, receiving software could parse that out as needed
by the simpler act of reading one line at a time.

And, of course, there are endless storage formats such as a CSV file or
serializing your list of objects to a file so that the next program can load
them in and operate from memory on all the ones it wants. The two file
solution may seem simpler but harks back to how some computing was done in
early days when list of objects might be handled by having multiple arrays
with each containing one aspect of the object and updating required
rememebreing to touch each array the same way.. That can still be a useful
technique when some operations being done in a vectoried manner might be
faster than an array of objects, but is more often a sign of poor code.






-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Grizzy Adams via Python-list
Sent: Friday, January 12, 2024 1:59 AM
To: Rich Shepard via Python-list <python-***@python.org>; Rich Shepard
<***@appl-ecosys.com>
Subject: Re: Extract lines from file, add to new files

Thursday, January 11, 2024 at 10:44, Rich Shepard via Python-list wrote:
Re: Extract lines from file, add to (at least in part)

>On Thu, 11 Jan 2024, MRAB via Python-list wrote:

>> From the look of it:
>> 1. If the line is empty, ignore it.
>> 2. If the line contains "@", it's an email address.
>> 3. Otherwise, it's a name.

If that is it all? a simple Grep would do (and save on the blank line)
--
https://mail.python.org/mailman/listinfo/python-list

[Rich Shepard]

On Fri, 12 Jan 2024, AVI GROSS via Python-list wrote:

> But is the solution a good one for some purpose? The two output files may
> end up being out of sync for all kinds of reasons. One of many "errors"
> can happen if multiple lines in a row do not have an "@" or a person's
> name does, for example. What if someone supplied more than one email
> address with a comma separator? This may not be expected but could cause
> problems.

Avi,

For my use 1) the salutation and email address (always with an '@') are
sequential and 2) I'm developing the script to extract both from the same
file.

Regards,

Rich

[Left Right]

To people discussing BNF:

The grammar language Python uses is *very* far from BNF. It's more
similar to PEG, but even then it's still quite far. Python's grammar
is just its own thing, which makes it harder to read, if you are
already familiar with other more popular formats.

I've also found bugs in Python parser before, so had this turned out
to be a real issue, this wouldn't have been the first time. There are
plenty of weird corners in Python grammar that allow unexpected
programs to parse (and sometimes even run!), and these are very often
connected to assignments, because, in general, assignments in Python
are very elaborate and hard to describe / conceptualize about. The
most popular example I've even seen used in coding interviews (which I
think is a silly gimmick, but that's kind of the whole point of a lot
of these interviews...) is:

x = [...]
for x[i] in x: print(i)

Which is not an assignment by itself, but the "weirdness" results from
the loop syntax sharing definitions with the "destructuring bind"
style of assignment (i.e. where the left-hand side can be an arbitrary
complex expression).

I was surprised, for example, to learn that "as" in "with_stmt" isn't
shared with "as" in "except_block" (so, from the grammar perspective,
these are two different keywords), and that asterisk in "except_block"
isn't shared with "star_target" (also weird, since you'd think these
should be the same thing). In general, and by and large, if you look
at Python's grammar there are many "weird" choices that it makes to
describe the language which seem counterintuitive to the programmer
who tries to learn the language from examples (i.e. context-depending
meaning of parenthesis, of asterisk, of period etc.) Having been
exposed to this, you'd start to expect that some of this weirdness
will eventually result in bugs, or at least in unexpected behavior.

----

Anyways. To the OP: I'm sorry to hijack your question. Below is the
complete program:

with (
open('example.txt', 'r') as e,
open('emails.txt', 'w') as m,
open('salutations.txt', 'w') as s,
):
for line in e:
if line.strip():
(m if '@' in line else s).write(line)

it turned out to be not quite the golfing material I was hoping for.
But, perhaps a somewhat interesting aspect of this program you don't
see used a lot in the wild is the parenthesis in the "with" head. So,
it's not a total write-off from the learning perspective. I.e. w/o
looking at the grammar, and had I have this code in a coding interview
question, I wouldn't be quite sure whether this code would work or
not: one way to interpret what's going on here is to think that the
expression inside parentheses is a tuple, and since tuples aren't
context managers, it wouldn't have worked (or maybe not even parsed as
"as" wouldn't be allowed inside tuple definition since there's no
"universal as-expression" in Python it's hard to tell what the rules
are). But, it turns out there's a form of "with" that has parentheses
for decoration purposes, and that's why it parses and works to the
desired effect.

Since it looks like you are doing this for educational reasons, I
think there's a tiny bit of value to my effort.

On Fri, Jan 12, 2024 at 8:08 AM Grizzy Adams via Python-list
<python-***@python.org> wrote:
>
> Thursday, January 11, 2024 at 10:44, Rich Shepard via Python-list wrote:
> Re: Extract lines from file, add to (at least in part)
>
> >On Thu, 11 Jan 2024, MRAB via Python-list wrote:
>
> >> From the look of it:
> >> 1. If the line is empty, ignore it.
> >> 2. If the line contains "@", it's an email address.
> >> 3. Otherwise, it's a name.
>
> If that is it all? a simple Grep would do (and save on the blank line)
> --
> https://mail.python.org/mailman/listinfo/python-list

[Greg Ewing]

On 13/01/24 12:11 am, Left Right wrote:
> x = [...]
> for x[i] in x: print(i)

I suspect you've misremembered something, because this doesn't
do anything surprising for me:

>>> x = [1, 2, 3]
>>> for x[i] in x: print(i)
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'i' is not defined

There's no destructuring going on here, just assignment to a
sequence item.

--
Greg

[Left Right]

> surprising for me:

Surprise is subjective, it's based on personal experience. Very few
languages allow arbitrary complex expressions in the same place they
allow variable introduction. The fact that "i" is not defined is
irrelevant to this example. Most programmers who haven't memorized
Python grammar by heart, but expect the language to behave similar to
the languages in the same category would be surprised this code is
valid (i.e. can be parsed), whether it results in error or not is of
no consequence.

> There's no destructuring going on here

I use the term "destructuring" in the same way Hyperspec uses it.
It's not a Python term. I don't know what you call the same thing in
Python. I'm not sure what you understand from it.

On Sat, Jan 13, 2024 at 12:37 AM Greg Ewing via Python-list
<python-***@python.org> wrote:
>
> On 13/01/24 12:11 am, Left Right wrote:
> > x = [...]
> > for x[i] in x: print(i)
>
> I suspect you've misremembered something, because this doesn't
> do anything surprising for me:
>
> >>> x = [1, 2, 3]
> >>> for x[i] in x: print(i)
> ...
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> NameError: name 'i' is not defined
>
> There's no destructuring going on here, just assignment to a
> sequence item.
>
> --
> Greg
> --
> https://mail.python.org/mailman/listinfo/python-list

[Greg Ewing]

On 13/01/24 1:45 pm, Left Right wrote:

> I use the term "destructuring" in the same way Hyperspec uses it.
> It's not a Python term. I don't know what you call the same thing in
> Python. I'm not sure what you understand from it.

I thought you meant what is usually called "unpacking" in Python. I
don't know anything about Hyperspec, so I don't know what it means
there.

The fact that i was being printed inside the loop made me think
that some deeper level of surprise was being intended, such as
the value of i somehow getting changed by the assignment.

--
Greg

[Left Right]

Actually, after some Web search. I think, based on this:
https://docs.python.org/3/reference/simple_stmts.html#grammar-token-python-grammar-augtarget
that in Python you call this "augmented assignment target". The term
isn't in the glossary, but so are many others.

On Sat, Jan 13, 2024 at 1:45 AM Left Right <***@gmail.com> wrote:
>
> > surprising for me:
>
> Surprise is subjective, it's based on personal experience. Very few
> languages allow arbitrary complex expressions in the same place they
> allow variable introduction. The fact that "i" is not defined is
> irrelevant to this example. Most programmers who haven't memorized
> Python grammar by heart, but expect the language to behave similar to
> the languages in the same category would be surprised this code is
> valid (i.e. can be parsed), whether it results in error or not is of
> no consequence.
>
> > There's no destructuring going on here
>
> I use the term "destructuring" in the same way Hyperspec uses it.
> It's not a Python term. I don't know what you call the same thing in
> Python. I'm not sure what you understand from it.
>
> On Sat, Jan 13, 2024 at 12:37 AM Greg Ewing via Python-list
> <python-***@python.org> wrote:
> >
> > On 13/01/24 12:11 am, Left Right wrote:
> > > x = [...]
> > > for x[i] in x: print(i)
> >
> > I suspect you've misremembered something, because this doesn't
> > do anything surprising for me:
> >
> > >>> x = [1, 2, 3]
> > >>> for x[i] in x: print(i)
> > ...
> > Traceback (most recent call last):
> > File "<stdin>", line 1, in <module>
> > NameError: name 'i' is not defined
> >
> > There's no destructuring going on here, just assignment to a
> > sequence item.
> >
> > --
> > Greg
> > --
> > https://mail.python.org/mailman/listinfo/python-list

[Chris Angelico]

On Sat, 13 Jan 2024 at 13:11, Left Right via Python-list
<python-***@python.org> wrote:
>
> Very few
> languages allow arbitrary complex expressions in the same place they
> allow variable introduction.

What do you mean by this? Most languages I've worked with allow
variables to be initialized with arbitrary expressions, and a lot of
languages allow narrowly-scoped variables.

ChrisA

[Greg Ewing]

On 13/01/24 3:14 pm, Chris Angelico wrote:
> On Sat, 13 Jan 2024 at 13:11, Left Right via Python-list
> <python-***@python.org> wrote:
>>
>> Very few
>> languages allow arbitrary complex expressions in the same place they
>> allow variable introduction.
>
> What do you mean by this? Most languages I've worked with allow
> variables to be initialized with arbitrary expressions, and a lot of
> languages allow narrowly-scoped variables.

I think he means that in some languages the for-loop target serves as
the declaration of a new variable, and as such has to be a bare name.

Python isn't like that -- the target of a for-statement is treated
exactly the same way as the lhs of an assignment. It's not scoped to the
loop.

BTW, the equivalent thing is valid in C too, so anyone familiar with C
is unlikely to be surprised by this either.

#include <stdio.h>

int x[10];
int i;

int main() {
i = 5;
for (x[i] = 0; x[i] < 10; x[i]++)
printf("%d\n", x[i]);
}

Output:

0
1
2
3
4
5
6
7
8
9

--
Greg

[2***@potatochowder.com]

On 2024-01-13 at 02:02:39 +0100,
Left Right via Python-list <python-***@python.org> wrote:

> Actually, after some Web search. I think, based on this:
> https://docs.python.org/3/reference/simple_stmts.html#grammar-token-python-grammar-augtarget
> that in Python you call this "augmented assignment target". The term
> isn't in the glossary, but so are many others.

The Python term, at least colloquially, is "tuple unpacking."

HTH.

[2***@potatochowder.com]

On 2024-01-13 at 11:34:29 +0100,
Left Right <***@gmail.com> wrote:

> > The Python term, at least colloquially, is "tuple unpacking."

That quote is from me. Please do preserve attributions.

> Well, why use colloquialism if there's a language specification? Also,
> there weren't any tuples used in my example, at least not explicitly
> (i could've been a tuple, but that wasn't specified).

According to the language specification,⁰ it's a "target list," and
there can be more than one target in that list.

The unpacking isn't really called anything, it's just the way Python
assignment works, all the way back to its earliest stages.¹

⁰ https://docs.python.org/3/reference/simple_stmts.html#assignment-statements,
¹ https://docs.python.org/release/1.4/ref/ref6.html#HDR2

[dn]

On 13/01/24 00:11, Left Right via Python-list wrote:
> To people discussing BNF:
>
> The grammar language Python uses is *very* far from BNF. It's more
> similar to PEG, but even then it's still quite far. Python's grammar
> is just its own thing, which makes it harder to read, if you are
> already familiar with other more popular formats.

Second time to ameliorate wording-dispute in this thread! The original
phrase was: "[modified] BNF". Some of us have worked with various forms
and evolutions of BNF since back in the days of COBOL-60 proposals, and
know it when we see it!

From the 'book of words': <<<The descriptions of lexical analysis and
syntax use a modified Backus–Naur form (BNF) grammar notation.>>>
https://docs.python.org/3/reference/introduction.html#notation

Yes it is hard to read - and even harder to learn-from; which is why
@Chris gave advice about preferring tutorials/text.

Just because there are other (more popular?) formats, doesn't make the
one used here 'wrong'. In the same way that Python code differs from
'the same' in other languages.

Putting it another way: if what Python is doing is wrong in your
opinion, you are in the wrong place (for you).

That is not to say that Python has everything 'right'. One of my own
bug-bears is very similar - that the string formatting 'mini-language'
(https://docs.python.org/3/library/string.html#formatspec) does not
follow the same rules about white-space as everything covered by the PEG
Parser.

BTW the PEG Parser is relatively new to Python. IIRC there was comment
at the time of its first application, that some 'other' areas of Python
might take a while to be converted-over.


> I've also found bugs in Python parser before, so had this turned out

Sorry, didn't recognise your email-handle - not that I'm a Python
Core-Dev, and pretty much ignore the "Ideas" list these days. Must have
missed your previous contributions...


> to be a real issue, this wouldn't have been the first time. There are
> plenty of weird corners in Python grammar that allow unexpected
> programs to parse (and sometimes even run!), and these are very often
> connected to assignments, because, in general, assignments in Python
> are very elaborate and hard to describe / conceptualize about. The
> most popular example I've even seen used in coding interviews (which I
> think is a silly gimmick, but that's kind of the whole point of a lot
> of these interviews...) is:
>
> x = [...]
> for x[i] in x: print(i)
>
> Which is not an assignment by itself, but the "weirdness" results from
> the loop syntax sharing definitions with the "destructuring bind"
> style of assignment (i.e. where the left-hand side can be an arbitrary
> complex expression).

You're right. (also about stupid 'interviewing' ideas) If someone asked
me this, I'd respond by asking if that was the standard of code they
work towards - and depending upon that answer would either walk-out or
refer the matter to a more senior manager!


In Python, everything is an object. As long as the LHS is a legal-object
which makes sense for the situation, it can be used.

Also, an identifier (whether x, i, or x[ i ]) should not only be
considered to be its own object, but is best regarded as a pointer to
some value. This is how we can have an 'immutable' tuple 'containing' a
mutable list (for example) - such that elements of that list may be
changed, despite being 'part of' an immutable construct!

Programs are read by people. If something is a "weirdness", then
chances-are it won't survive a CodeReview/a professional team's
expected-standard. Not limited to Python-code!


> I was surprised, for example, to learn that "as" in "with_stmt" isn't
> shared with "as" in "except_block" (so, from the grammar perspective,
> these are two different keywords), and that asterisk in "except_block"
> isn't shared with "star_target" (also weird, since you'd think these
> should be the same thing). In general, and by and large, if you look
> at Python's grammar there are many "weird" choices that it makes to
> describe the language which seem counterintuitive to the programmer
> who tries to learn the language from examples (i.e. context-depending
> meaning of parenthesis, of asterisk, of period etc.) Having been
> exposed to this, you'd start to expect that some of this weirdness
> will eventually result in bugs, or at least in unexpected behavior.

You're right. It is potentially confusing when the same word/symbol is
used in different contexts.

I've heard similar questions from learners, but not had anyone trying to
mis-use something extrapolating from how the 'same' is used elsewhere. YMMV!

It's the context part that's important to remember. If someone calls you
"mate", that has different connotations depending upon whether you're
friends, you're on a Navy ship, or in a more intimate situation - indeed
there are some cultures in which the word "mate" is not used to mean
'friend' at all. Which is (more) right? Which wrong?

Perhaps you're aiming for, or even used to, a more perfect and
predictable language?


> ----
>
> Anyways. To the OP: I'm sorry to hijack your question. Below is the
> complete program:
>
> with (
> open('example.txt', 'r') as e,
> open('emails.txt', 'w') as m,
> open('salutations.txt', 'w') as s,
> ):
> for line in e:
> if line.strip():
> (m if '@' in line else s).write(line)

Please see responses elsewhere which say why this sort of thing, whilst
possible and 'easy', is not recommendable.


> it turned out to be not quite the golfing material I was hoping for.
> But, perhaps a somewhat interesting aspect of this program you don't
> see used a lot in the wild is the parenthesis in the "with" head. So,
> it's not a total write-off from the learning perspective. I.e. w/o
> looking at the grammar, and had I have this code in a coding interview
> question, I wouldn't be quite sure whether this code would work or
> not: one way to interpret what's going on here is to think that the
> expression inside parentheses is a tuple, and since tuples aren't
> context managers, it wouldn't have worked (or maybe not even parsed as
> "as" wouldn't be allowed inside tuple definition since there's no
> "universal as-expression" in Python it's hard to tell what the rules
> are). But, it turns out there's a form of "with" that has parentheses
> for decoration purposes, and that's why it parses and works to the
> desired effect.

Again, context! All that is important 'here' is how to 'link' the
file-descriptor with an identifier. Similarly, whilst we could write:

a, b, c = 1, 2, 3

(and BTW that is legal Python - for anyone seeing such for the first time)
and whilst it is shorter (and I've been known to write such), the need
to read carefully in order to pair-up the relative positions make it
less readable than

a = 1; b = 2; c = 3

(and some would argue, quite reasonably, that it would be better were
they on separate lines)

Similarly, many dev.teams have a 'standard' which suggests that once a
function/method has three or more arguments, relative-positioning should
go out-the-window, in favor of named-arguments. This speeds
comprehension and reduces errors.

In the original mental-model, the difficulty was which file-descriptor
would be paired with which file (previously described). The multiple
as-s make it more readable and more comprehensible.


> Since it looks like you are doing this for educational reasons, I
> think there's a tiny bit of value to my effort.

That's what we're (all) here for!
(and not forgetting that the OP described a skill-level well below that
of most of this post and your question, which enabled (and deserved,
IMHO) appropriate respect).

--
Regards,
=dn

[Chris Angelico]

On Sun, 14 Jan 2024 at 14:43, dn via Python-list <python-***@python.org> wrote:
> Similarly, whilst we could write:
>
> a, b, c = 1, 2, 3
>

I would only do this when it aligns particularly well with the
algorithm being implemented. For example, you could start a Fibonacci
evaluator with "a, b = 0, 1". Otherwise, there's not all that much
reason to unpack three constants in this way.

(Though I am much more likely to use multiple initialization to set a
bunch of things to the SAME value, lilke "a = b = c = 0".)

[Chris Angelico]

On Sun, 14 Jan 2024 at 23:28, Left Right <***@gmail.com> wrote:
> Having worked with a bunch of different grammar languages, the one
> used for Python isn't a recognizable BNF derivative.

That might possibly be because it isn't? It's not BNF. It's PEG. Or
are you a long way behind the times?

> For example, you may say "functions in Python are
> objects", but you cannot put a function definition in the head of the
> for loop clause.

What do you mean?

for x in lambda: ...:
...

Perfectly grammatical.

ChrisA

[Chris Angelico]

On Mon, 15 Jan 2024 at 00:27, Left Right <***@gmail.com> wrote:
>
> > What do you mean?
> >
> > for x in lambda: ...:
> > ...
> >
> > Perfectly grammatical.
>
> 1. You put the lambda definition in the wrong place (it should be in
> the left-hand side, or as Python calls it "star_targets", but you put
> it into "star_expressions", which would be where the right-hand side
> is drawn from).
> 2. You used what Python calls "lambdadef" in place of what Python
> calls "function_def". I.e. lambda definition and function definition
> are two different things, at least as far as grammar is considered.
>
> So, you solved a different problem.

You said function. I made a function. You said "head of a for loop
clause". I put it there. Problem was underspecified.

But if you're trying to tell me that a def statement should be a valid
assignment target, I don't know what you're smoking, but I want you to
keep it a long way away from me. Can you name ANY language in which
that would make the slightest bit of sense?

ChrisA

[a***@gmail.com]

It can be worth considering why a language is designed or altered in certain
ways to see if there was a tradeoff that made it seem worthwhile or easier
than some other choice.

Python grew and there was regular pressure to add keywords which might break
existing programs. So, yes, sometimes, a keyword was re-used in a different
context. And, yes, it was not originally conceived in a purely object
oriented context.

If you wanted to start over and built a new language very similar to python,
you might indeed make other choices now that seem more seamlessly to fit
together. You could set aside and reserve hundreds of keywords or some way
to extend keywords by insisting anything staring with "key_" cannot be used
in a variable name. You might design all the main objects supported to all
support a function that provides a length as well as every other method
needed so it looks purely object oriented.

But perhaps that would make it a tad harder to program it using other ways.
As an example, I can ask some sort program to order the results by the
length of items by passing it the function that does lengths as an argument.
If instead all we had was a method, that might be a bit different and
perhaps someone would simply make a tiny function that when called, invoked
the method.

So, we have a hybrid of sorts and have to live with it, warts and all, and
some of the warts may be seen by some as beauty marks.




-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Chris Angelico via Python-list
Sent: Sunday, January 14, 2024 7:32 AM
To: python-***@python.org
Subject: Re: Extract lines from file, add to new files

On Sun, 14 Jan 2024 at 23:28, Left Right <***@gmail.com> wrote:
> Having worked with a bunch of different grammar languages, the one
> used for Python isn't a recognizable BNF derivative.

That might possibly be because it isn't? It's not BNF. It's PEG. Or
are you a long way behind the times?

> For example, you may say "functions in Python are
> objects", but you cannot put a function definition in the head of the
> for loop clause.

What do you mean?

for x in lambda: ...:
...

Perfectly grammatical.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

[Rich Shepard]

On Fri, 12 Jan 2024, Rich Shepard via Python-list wrote:

> For my use 1) the salutation and email address (always with an '@') are
> sequential and 2) I'm developing the script to extract both from the same
> file.

I've looked at my Python books "Python Crash Course," "Effective Python,"
and "Python Tricks The Book" as well as web pages in my searches without
finding the answer to what may be a simple question: how to specify a
variable in one file that has its values in another file.

Specifically, how to I designate the salutation holder in the message file
and pass it the name value from the name/email address file?

If this explanation is not sufficiently clear I'll re-write it. :-)

TIA,

Rich

[Stefan Ram]

Rich Shepard <***@appl-ecosys.com> writes: how to specify a
>variable in one file that has its values in another file.

In the first file, you can write

variable = eval( open( "otherfile" ).read() )

, and the value for "variable" will be taken from "otherfile".

$ echo 3.14 >otherfile
$ echo "v = eval( open( 'otherfile' ).read() ); print( v )" > thisfile
$ python3 thisfile
3.14
$

[Rich Shepard]

On Mon, 29 Jan 2024, ***@online.de wrote:

> Have you read "https://docs.python.org/3/library/io.html#module-io"?

Dieter,

No, I hadn't ... but I am reading it now.

Many thanks,

Rich

[d***@online.de]

Rich Shepard wrote at 2024-1-29 08:15 -0800:
> ...
>If this explanation is not sufficiently clear I'll re-write it. :-)

Have you read "https://docs.python.org/3/library/io.html#module-io"?

[Rich Shepard]

On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:

> No, I hadn't ... but I am reading it now.

Perhaps I missed the answer to my question when reading the io module. It
explains how to open/write/read files of text and binary data, not passing
a variable's value from one file to a place-keeper in another file.

I'll keep searching for a solution.

Rich

[Jon Ribbens]

On 2024-01-29, Rich Shepard <***@appl-ecosys.com> wrote:
> On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:
>> No, I hadn't ... but I am reading it now.
>
> Perhaps I missed the answer to my question when reading the io module. It
> explains how to open/write/read files of text and binary data, not passing
> a variable's value from one file to a place-keeper in another file.

Why would it contain a recipe for performing some specific task that
appears unique to you?

If you want any meaningful help you'll need to explain more precisely
what it is that you're trying to achieve and why.

[Rich Shepard]

On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:

> I'll keep searching for a solution.

IIRC, someone here pointed me to <https://realpython.com/python-send-email/>
and I forgot about it ... until now.

Regards,

Rich

[a***@gmail.com]

Rich,

You got an overly general reply to a question many of us may not understand.

You have not hinted at how the two files are organized, perhaps with an
example.

There are several general solutions that may apply. Some involve reading in
both files into data structures and perhaps linking them together in some
way such as a data.frame or binary tree. You can then process individual
request in memory/

The second should be straightforward as long as text is text. If the first
file tells you to search for XYZ then you search the second file for XYZ and
read in whatever is associated with it and do your thing.

Without a bit more specific detail, you may not get more than a suggestion
as to how to read in files.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Rich Shepard via Python-list
Sent: Monday, January 29, 2024 12:38 PM
To: python-***@python.org
Subject: RE: Extract lines from file, add to new files

On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:

> No, I hadn't ... but I am reading it now.

Perhaps I missed the answer to my question when reading the io module. It
explains how to open/write/read files of text and binary data, not passing
a variable's value from one file to a place-keeper in another file.

I'll keep searching for a solution.

Rich
--
https://mail.python.org/mailman/listinfo/python-list

[Rich Shepard]

On Mon, 29 Jan 2024, ***@gmail.com wrote:

> There are several general solutions that may apply. Some involve reading
> in both files into data structures and perhaps linking them together in
> some way such as a data.frame or binary tree. You can then process
> individual request in memory/

Avi,

I found several web pages describing how to use the python email library and
tools to send individual or multiple email messages. I'll learn how to do
this based on a couple of detailed examples.

Thanks,

Rich

[dn]

On 30/01/24 05:15, Rich Shepard via Python-list wrote:
> On Fri, 12 Jan 2024, Rich Shepard via Python-list wrote:
>
>> For my use 1) the salutation and email address (always with an '@') are
>> sequential and 2) I'm developing the script to extract both from the same
>> file.
>
> I've looked at my Python books "Python Crash Course," "Effective Python,"
> and "Python Tricks The Book" as well as web pages in my searches without
> finding the answer to what may be a simple question: how to specify a
> variable in one file that has its values in another file.
>
> Specifically, how to I designate the salutation holder in the message file
> and pass it the name value from the name/email address file?
>
> If this explanation is not sufficiently clear I'll re-write it. :-)


It seems clear - but maybe we (all) misunderstand?

The books don't cover it - but is there a reason why they don't?


(Thunderbird did not thread this message with earlier ones, and it seems
some time has passed/memory is fading - but hopefully have remembered
content)


As said previously, the idea of two physical-files containing
logically-related data (without any form of cross-reference between) is
bad design.

In the current file, there is one physical structure and each person's
details are logically-related by proximity. A long way from ideal, but
workable (as you've described).

However, once split into two files, there is no way to guarantee that
the two logically-related data-items (name and address) will continue to
be related by their physical position in the respective files. Worse:
whilst it would seem apparent that "Alice" from the names file might be
related to the address "***@domain.tld", how could one know if "Bob"
actually corresponds to "list-***@domain.tld"?

This is why dicts, databases, etc, offer keys (as labels for
data-items/dependent components)!


After a quick look at Eric's Crash Course, yes, his files-intro example
(digits of pi) is unlikely to have any practical reality (and I work
with statisticians and quants!). However, at the end of that chapter
(10), there is mention of JSON files. A JSON version of the existing
single-file structure will provide human-readable labeling of
data-items, give better separation between individuals' data, and show
how name and address are linked. Recommend solving 'the problem' that
way! (as previously discussed by others 'here', IIRC)

Slatkin's Effective Python doesn't seem to discuss the basics of files
at all (presumes such knowledge of the reader). It dives into important,
but rather technical discussions, comparing strings and bytes - somewhat
beyond the complexity-level of this discussion. That book does however
include discussions such as "Prefer Multiple Assignment Unpacking over
Indexing" (Item 6 - also points to Item 19) where relative-positioning
(indexing in other words) is advised-against.


If you wish to persist with this two-file structure, please see earlier
responses (again, IIRC) and discussion of file-merge operations. As
these date back to mainframes and the days of storing files on mag-tape,
I'd be surprised to see them discussed in 'modern' texts. However, the
principle is: read a record from each file, do-the-business, read the
next 'pair' of physically-related records, rinse-and-repeat.

If you require further assistance: how about showing a couple of
relevant lines of the data-file(s) and the pertinent parts of the code,
along with a description of what's going-wrong or not making sense to you?

--
Regards,
=dn

[Grant Edwards]

On 2024-01-29, Rich Shepard via Python-list <python-***@python.org> wrote:
> On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:
>
>> No, I hadn't ... but I am reading it now.
>
> Perhaps I missed the answer to my question when reading the io module. It
> explains how to open/write/read files of text and binary data, not passing
> a variable's value from one file to a place-keeper in another file.

It's not at all clear (to me) what you're asking about. When you talk
about "files" are you referring to data files? Python modules within a
single program? Seperate Python programs? Something else?

The phrase "place-keeper in another file" sounds a bit like you're
trying to do templating. There are many, many ways to do templating in
Python -- ranging from literal 'f-strings' to powerful templating
engines that are used to construct entire web sites:

https://www.google.com/search?q=python+templating

https://docs.python.org/3/tutorial/inputoutput.html#tut-f-strings

https://en.wikipedia.org/wiki/Jinja_(template_engine)

[Thomas Passin]

On 1/29/2024 11:15 AM, Rich Shepard via Python-list wrote:
>> For my use 1) the salutation and email address (always with an '@') are
>> sequential and 2) I'm developing the script to extract both from the same
>> file.
>
> I've looked at my Python books "Python Crash Course," "Effective Python,"
> and "Python Tricks The Book" as well as web pages in my searches without
> finding the answer to what may be a simple question: how to specify a
> variable in one file that has its values in another file.
>
> Specifically, how to I designate the salutation holder in the message file
> and pass it the name value from the name/email address file?
>
> If this explanation is not sufficiently clear I'll re-write it. :-)
>
> TIA,
>
> Rich

I'm assuming this is a continuation of a previous thread about working
with alternate lines with salutation and address, and I assume you've
got that worked out.

If you aren't going to use one or another existing template system,
perhaps the easiest is to use unique strings in the message file. For
example:

Dear __##so-and-so##__:
Please don't write this message off as mere spam.
Respectfully, Rich

Then you just do a replace of the unique string by the salutation. Don't
change the original (i.e., template), make the changes to a copy that
you will output.

[a***@gmail.com]

It can be quite frustrating figuring out what someone wants, Grant,
especially when they just change it.

It is worse when instead of starting a new thread with an appropriate
subject line, it continues and old one that was also frustrating to
understand.

It sounds though like another attempt to do something perhaps a different
way. Both attempts seem to be to use some form of storage of a set of email
addresses plus other info like a name that can be used to make a customized
email.

Frankly, this should have been fairly easy to do without so much back and
forth. I don't care how the email is actually sent, but the rest could have
been done any number of ways such as storing the data as rows in a CSV file
or saved using JSON format and so on. It was never made clear why two files
were needed and then somehow linked and searched.

If the goal is to be able to search for something like a name and THEN find
an email address, that seems quite trivial if they are I the same file in
some format. If the number of items is small, reading it all in should not
be a big deal and you can use a regular expression or other method to locate
the entry you want and extract the additional info. If you have lots of
data, reading line after line may be less useful than just using a database
and a query.

One way to stop feeling frustrated is to stop reading the thread.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Grant Edwards via Python-list
Sent: Monday, January 29, 2024 5:54 PM
To: python-***@python.org
Subject: Re: Extract lines from file, add to new files

On 2024-01-29, Rich Shepard via Python-list <python-***@python.org> wrote:
> On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:
>
>> No, I hadn't ... but I am reading it now.
>
> Perhaps I missed the answer to my question when reading the io module. It
> explains how to open/write/read files of text and binary data, not passing
> a variable's value from one file to a place-keeper in another file.

It's not at all clear (to me) what you're asking about. When you talk
about "files" are you referring to data files? Python modules within a
single program? Seperate Python programs? Something else?

The phrase "place-keeper in another file" sounds a bit like you're
trying to do templating. There are many, many ways to do templating in
Python -- ranging from literal 'f-strings' to powerful templating
engines that are used to construct entire web sites:

https://www.google.com/search?q=python+templating

https://docs.python.org/3/tutorial/inputoutput.html#tut-f-strings

https://en.wikipedia.org/wiki/Jinja_(template_engine)

--
https://mail.python.org/mailman/listinfo/python-list

[Rich Shepard]

On Mon, 29 Jan 2024, Thomas Passin via Python-list wrote:

> If you aren't going to use one or another existing template system,
> perhaps the easiest is to use unique strings in the message file. For
> example:
>
> Dear __##so-and-so##__:
> Please don't write this message off as mere spam.
> Respectfully, Rich
>
> Then you just do a replace of the unique string by the salutation. Don't
> change the original (i.e., template), make the changes to a copy that you
> will output.

My script is not a web application, but an emailer that allows me to contact
clients and prospective clients. From the command line on a linux host.
Using the python smtplib and mail modules.

Rich

[Jon Ribbens]

On 2024-01-30, Rich Shepard <***@appl-ecosys.com> wrote:
> On Mon, 29 Jan 2024, Thomas Passin via Python-list wrote:
>
>> If you aren't going to use one or another existing template system,
>> perhaps the easiest is to use unique strings in the message file. For
>> example:
>>
>> Dear __##so-and-so##__:
>> Please don't write this message off as mere spam.
>> Respectfully, Rich
>>
>> Then you just do a replace of the unique string by the salutation. Don't
>> change the original (i.e., template), make the changes to a copy that you
>> will output.
>
> My script is not a web application, but an emailer that allows me to contact
> clients and prospective clients. From the command line on a linux host.
> Using the python smtplib and mail modules.

lol good luck with that.

[a***@gmail.com]

I deleted the contents of the message so I can avoid both of the deadly
sins of top posting and bottom posting and chance committing the sin of
replying without any context.

Of course, I am only replying to Jon wishing a real or feigned good luck to
the OP.

But seriously, the OP, AKA Rich, is making clear that he is making a tool
for his own use. It sounds like he wants to maintain a data repository of
his own with some info about his clients and then have the ability to
specify a name and pop up an email directed to them, or something along
those lines.

Without further info, this sounds like what quite a few popular and even
free mailers already do to some extent as you have an associated address
book. Some start doing possible matches as you type. I hazard Rich is not
using something this simple as he does seem to have some form-letter merge
or similar functionality in mind, such as automating the seemingly mandatory
"Dear XXX" salutation.

But if he is the creator and maintainer of his client data and chooses not
to use one of many available applications, then it seems he already has
reasons he wants a particular design for the data and simply wants us to
help him use it the way he wants. That could be communicated a bit more
clearly but after many messages back and forth, I have not exactly
understood it.

In my opinion, having created all kinds of mailers over the years, sometimes
the hard way, This strikes me as not really being about what mailing
functionality exists at all. As a general rule, you first create supporting
parts for your mail then pass them along to functionality that assembles the
mail as a set of headers and a body and dispatches it.

You as the programmer need to supply a body, tell it who to put on TO/CC/BB
lines, perhaps provide a subject, perhaps specify attachments, and off you
go. The exact methods will differ.

But what Rich presumably needs to do is have his program interact with him
in specifying who he wants to mail to and then looking it up in whatever
file arrangement it contains. If he also wants to use other parts such as a
human name or address inside the body of the text, his program needs to
merge the text he supplies along with extracted parts of the data.

Or is that not what he wants? The above could be quite straightforward and I
recall doing things like this with a simple shell script in UNIX.
Specifically, I created a text file where I recorded info for each person in
some format like

NAME|PHONE|EMAIL|COMMENT

Then to search for someone, you could use something like grep to find a name
by anchoring to the beginning or ending with the "|" so it does not match
the text in another field such as email or address, and use other utilities
ranging from cut to awk and getting the parts you want into variables and
then interpolate them into a message template and so on.

Of course, doing it in python is a good way to go too and should not be hard
once it is decided how to store the data. But again, this is re-inventing
things that others have already done.

The python modules include many ways to store modular data. The books I have
read mention them all the time. Pick one. And, yes, you can choose to
maintain two files if that design works for you. Consider some storage
method that stores data in sections like:

[NAME one]
First: whatever
Email: whatever

[NAME two]
First: ...
Email: ...

There are specific formats along these lines and you can get python modules
that you ask for "NAME one" and it reads the file until it finds a section
as "[NAME one]" or not. If found, it returns the variables/values right
below it. So your two step algorithm may consist of two files with one file
containing just names, perhaps to use a grep functionality on. Some of those
names will have a matching section like the above somewhere in the other
file.

So if you want to send mail to "Jo" then your program may search for all
names starting with "Jo" and offer you "John Smith" and perhaps also
"Joachim Martillo". The program takes whichever full name(s) you then select
and calls a function that uses that full name to search the second file to
find an exact match and returns what it finds there such as an email
address.

But unless you have lots of contacts, as already discussed, there are far
easier ways to do things in a more brute force way. Take a one-line per
entry format such as a CSV or TSV and extract whatever column contains the
name as needed to do the first search. Yes, this tends to mean reading the
entire file.

And, for the record, I am not a fan of hiding replies at the bottom except
for short messages. I prefer to use some combination of in-line if
addressing many points in the original and mainly the top with perhaps a
preface explaining what is being addressed. The reader is usually capable of
digging below if they want to know more. But this is a more religious war
having nothing to do with python specifically.

My frustration is that I often want to help someone and wish the problem was
stated in a way that made that doable. I do sympathize with Rich as figuring
out which details to focus on or to omit is not trivial. I may well be
wrong, but it sounds like his request could be focused on how to use
supporting files for an email application, or if stated clearly, as he
tried, about how to store data in two files and be able to extract what you
need as if it was all in one file.

But what may not be easy to get through is the experience of years from when
you often stored data in multiple arrays and had to be careful to update
them all at once, to more modern methods using structs or classes to hold
more or all aspects of a related object as a unit. In this context, some may
simply read in all the data from a single file into something like a list of
objects (or a numpy/pandas variant) and have search and manipulation
abilities to find and change the right ones. The only diddling with files
that is needed is once to read them into the structure (such as JSON) and
perhaps once to write it back out with changes.

Good luck, once you decide on a particular method and flesh that out.

[Rich Shepard]

On Tue, 30 Jan 2024, AVI GROSS via Python-list wrote:

> But seriously, the OP, AKA Rich, is making clear that he is making a tool
> for his own use. It sounds like he wants to maintain a data repository of
> his own with some info about his clients and then have the ability to
> specify a name and pop up an email directed to them, or something along
> those lines.

Close, Avi.

I have no issues sending messages to single individuals or mailing lists. I
want to send the same message to several individuals at one time, which I've
done -- without individual salutations -- for 30 years using a bash script
and mailx.

As I replied to Thomas on the list, I've downloaded 11 PDF docs from the Web
(and a useful book on the Python3 standard library) and will start reading
and learning from them today. I expect to find answers to my few remaining
questions in these docs.

Regards,

Rich

[Larry Martell]

On Tue, Jan 30, 2024 at 1:13 AM AVI GROSS via Python-list
<python-***@python.org> wrote:
>
> It can be quite frustrating figuring out what someone wants, Grant,
> especially when they just change it.
>
> It is worse when instead of starting a new thread with an appropriate
> subject line, it continues and old one that was also frustrating to
> understand.

Is it worse than top posting?

[Thomas Passin]

On 1/30/2024 8:37 AM, Rich Shepard via Python-list wrote:
> On Mon, 29 Jan 2024, Thomas Passin via Python-list wrote:
>
>> If you aren't going to use one or another existing template system,
>> perhaps the easiest is to use unique strings in the message file. For
>> example:
>>
>> Dear __##so-and-so##__:
>>   Please don't write this message off as mere spam.
>>   Respectfully, Rich
>>
>> Then you just do a replace of the unique string by the salutation. Don't
>> change the original (i.e., template), make the changes to a copy that you
>> will output.
>
> My script is not a web application, but an emailer that allows me to
> contact
> clients and prospective clients. From the command line on a linux host.
> Using the python smtplib and mail modules.
>
> Rich

Fine, my toy example will still be applicable. But, you know, you
haven't told us enough to give you help. Do you want to replace text
from values in a file? That's been covered. Do you want to send the
messages using those libraries? You haven't said what you don't know
how to do. Something else? What is it that you want to do that you
don't know how?

[Rich Shepard]

On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:

> Fine, my toy example will still be applicable. But, you know, you haven't
> told us enough to give you help. Do you want to replace text from values
> in a file? That's been covered. Do you want to send the messages using
> those libraries? You haven't said what you don't know how to do. Something
> else? What is it that you want to do that you don't know how?

Thomas,

For 30 years I've used a bash script using mailx to send messages to a list
of recipients. They have no salutation to personalize each one. Since I want
to add that personalized salutation I decided to write a python script to
replace the bash script.

I have collected 11 docs explaining the smtplib and email modules and
providing example scripts to apply them to send multiple individual messages
with salutations and attachments.

Today I'm going to be reading these. They each recommend using .csv input
files for names and addresses. My first search is learning whether I can
write a single .csv file such as:
"name1","address1"
"mane2","address2"
which I believe will work; and by inserting at the top of the message block
Hi, {yourname}
the name in the .csv file will replace the bracketed place holder.

Still much to learn and the batch of downloaded PDF files should educate me.

Regards,

Rich

[Karsten Hilbert]

> For 30 years I've used a bash script using mailx to send messages to a list
> of recipients. They have no salutation to personalize each one. Since I want
> to add that personalized salutation I decided to write a python script to
> replace the bash script.

Why not foxus on just the part you think you are better off using python, namely
personalization ?

Create personalized files and send them with your trusted mailx solution ?

That'll take out wrestling with smptlib et al.

Karsten

[Rich Shepard]

On Tue, 30 Jan 2024, Karsten Hilbert wrote:

> Why not foxus on just the part you think you are better off using python,
> namely personalization ?
>
> Create personalized files and send them with your trusted mailx solution ?

Karsten,

Too much time. And while mailx accepts the '-a' option for attachments but
has none for individual salutations.

Regards,

Rich

[Karsten Hilbert]

> > Why not foxus on just the part you think you are better off using python,
> > namely personalization ?
> >
> > Create personalized files and send them with your trusted mailx solution ?
>
> Karsten,
>
> Too much time. And while mailx accepts the '-a' option for attachments but
> has none for individual salutations.

It doesn't need to. It just sends the (pre-personalized-by-Python) mail files.

Karsten

[Rich Shepard]

On Tue, 30 Jan 2024, Karsten Hilbert wrote:

> It doesn't need to. It just sends the (pre-personalized-by-Python) mail files.

Karsten,

In which case, I might as well have Python format and send the messages. :-)

Regards,

Rich

[Karsten Hilbert]

> On Tue, 30 Jan 2024, Karsten Hilbert wrote:
>
> > It doesn't need to. It just sends the (pre-personalized-by-Python) mail files.
>
> Karsten,
>
> In which case, I might as well have Python format and send the messages. :-)

Certainly. But it seems you are wrestling with Python. Might as well reduce the attack surface.

Karsten

[a***@gmail.com]

Rich,

You may want to broaden your perspective a bit when people make suggestions.

Karsten did not spell out a full design and should not need to.

But consider this as a scenario.

You want to send (almost) the same message to one or more recipients.

So call a program, perhaps some variant on a shell script, that does some
prep work such as maybe creating a temporary or working directory/folder.
Had one copy of your message ready in a file somewhere, Have a way to get a
list of recipients intended and the file or files containing enough info to
link email addresses to human names and anything else such as their
preferred pronoun or address.

Now have the script call your super-duper python program with enough info so
it can find the folder to put amended COPIES of your letter into as well as.
Perhaps the email address intended in the filename or whatever works for
you.

Your program will then simply identify each email recipient you want and
look up the other info and prepend the customized salutation, or make
substitutions in the template and write out a new file in the designated
folder with perhaps the email address as the filename.

When your loop ends, exit the python program with success, or perhaps report
some failure.

The shell script now resumes by checking the exit status and if OK,
continuing to enter the folder and loop on all file contests and invoke the
functionality to send each copied/enhanced file to the intended recipient.
If you also need to support attachments, you can figure out how to attach
the same ones to each as I assume those are not changed for each recipient.

It may keep track of how many worked or failed and eventually clear out the
files and perhaps the folder and you are done.

This is NOT a required way to do it but for what sounds like a limited
personal project, it should work well enough and have you do the limited
amount of work you need in Python.

Having said that, you can likely also easily do everything without python
and I have written some huge shell scripts in my time to do way more complex
things. But learning how to do things like this well in python can be time
well spent as long as you don't tackle too much at a time and get
overwhelmed.




-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Rich Shepard via Python-list
Sent: Tuesday, January 30, 2024 12:53 PM
To: python-***@python.org
Subject: Re: Aw: Re: Extract lines from file, add to new files

On Tue, 30 Jan 2024, Karsten Hilbert wrote:

> Why not foxus on just the part you think you are better off using python,
> namely personalization ?
>
> Create personalized files and send them with your trusted mailx solution ?

Karsten,

Too much time. And while mailx accepts the '-a' option for attachments but
has none for individual salutations.

Regards,

Rich
--
https://mail.python.org/mailman/listinfo/python-list

[Mats Wichmann]

On 1/30/24 14:46, AVI GROSS via Python-list wrote:
> Rich,
>
> You may want to broaden your perspective a bit when people make suggestions.
>
> Karsten did not spell out a full design and should not need to.
>
> But consider this as a scenario.
>
> You want to send (almost) the same message to one or more recipients.
>
> So call a program, perhaps some variant on a shell script, that does some
> prep work such as maybe creating a temporary or working directory/folder.
> Had one copy of your message ready in a file somewhere, Have a way to get a
> list of recipients intended and the file or files containing enough info to
> link email addresses to human names and anything else such as their
> preferred pronoun or address.

I'd say based on the bits of the problem description I *have* absorbed,
which almost certainly isn't all of them, there's a fairly basic
capability, not terribly often used in my experience, that might be of
some use:

https://docs.python.org/3/library/string.html#template-strings

[Thomas Passin]

On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>
>> Fine, my toy example will still be applicable. But, you know, you haven't
>> told us enough to give you help. Do you want to replace text from values
>> in a file? That's been covered. Do you want to send the messages using
>> those libraries? You haven't said what you don't know how to do.
>> Something
>> else? What is it that you want to do that you don't know how?
>
> Thomas,
>
> For 30 years I've used a bash script using mailx to send messages to a list
> of recipients. They have no salutation to personalize each one. Since I
> want
> to add that personalized salutation I decided to write a python script to
> replace the bash script.
>
> I have collected 11 docs explaining the smtplib and email modules and
> providing example scripts to apply them to send multiple individual
> messages
> with salutations and attachments.

If I had a script that's been working for 30 years, I'd probably just
use Python to do the personalizing and let the rest of the bash script
do the rest, like it always has. The Python program would pipe or send
the personalized messages to the rest of the bash program. Something in
that ballpark, anyway.

> Today I'm going to be reading these. They each recommend using .csv input
> files for names and addresses. My first search is learning whether I can
> write a single .csv file such as:
> "name1","address1"
> "mane2","address2"
> which I believe will work; and by inserting at the top of the message block
> Hi, {yourname}
> the name in the .csv file will replace the bracketed place holder
If the file contents are going to be people's names and email addresses,
I would just tab separate them and split each line on the tab. Names
aren't going to include tabs so that would be safe. Email addresses
might theoretically include a tab inside a quoted name but that would be
extremely obscure and unlikely. No need for CSV, it would just add
complexity.

data = f.readlines()
for d in data:
name, addr = line.split('\t') if line.strip() else ('', '')

> Still much to learn and the batch of downloaded PDF files should educate
> me.
>
> Regards,
>
> Rich

[a***@gmail.com]

Thomas, on some points we may see it differently.

Some formats can be done simply but are maybe better done in somewhat
standard ways.

Some of what the OP has is already tables in a database and that can
trivially be exported into a CSV file or other formats like your TSV file
and more. They can also import from there. As I mentioned, many spreadsheets
and all kinds of statistical programs tend to support some formats making it
quite flexible.

Python has all kinds of functionality, such as in the pandas module, to read
in a CSV or write it out. And once you have the data structure in memory, al
kinds of queries and changes can be made fairly straightforwardly. As one
example, Rich has mentioned wanting finer control in selecting who gets some
version of the email based on concepts like market segmentation. He already
may have info like the STATE (as in Arizona) in his database. He might at
some point enlarge his schema so each entry is placed in one or more
categories and thus his CSV, once imported, can do the usual tasks of
selecting various rows and columns or doing joins or whatever.

Mind you, another architecture could place quite a bit of work completely on
the back end and he could send SQL queries to the database from python and
get back his results into python which would then make the email messages
and pass them on to other functionality to deliver. This would remove any
need for files and just rely on the DB.

There as as usual, too many choices and not necessarily one best answer. Of
course if this was a major product that would be heavily used, sure, you
could tweak and optimize. As it is, Rich is getting a chance to improve his
python skills no matter which way he goes.



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Thomas Passin via Python-list
Sent: Tuesday, January 30, 2024 10:37 PM
To: python-***@python.org
Subject: Re: Extract lines from file, add to new files

On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>
>> Fine, my toy example will still be applicable. But, you know, you haven't
>> told us enough to give you help. Do you want to replace text from values
>> in a file? That's been covered. Do you want to send the messages using
>> those libraries? You haven't said what you don't know how to do.
>> Something
>> else? What is it that you want to do that you don't know how?
>
> Thomas,
>
> For 30 years I've used a bash script using mailx to send messages to a
list
> of recipients. They have no salutation to personalize each one. Since I
> want
> to add that personalized salutation I decided to write a python script to
> replace the bash script.
>
> I have collected 11 docs explaining the smtplib and email modules and
> providing example scripts to apply them to send multiple individual
> messages
> with salutations and attachments.

If I had a script that's been working for 30 years, I'd probably just
use Python to do the personalizing and let the rest of the bash script
do the rest, like it always has. The Python program would pipe or send
the personalized messages to the rest of the bash program. Something in
that ballpark, anyway.

> Today I'm going to be reading these. They each recommend using .csv input
> files for names and addresses. My first search is learning whether I can
> write a single .csv file such as:
> "name1","address1"
> "mane2","address2"
> which I believe will work; and by inserting at the top of the message
block
> Hi, {yourname}
> the name in the .csv file will replace the bracketed place holder
If the file contents are going to be people's names and email addresses,
I would just tab separate them and split each line on the tab. Names
aren't going to include tabs so that would be safe. Email addresses
might theoretically include a tab inside a quoted name but that would be
extremely obscure and unlikely. No need for CSV, it would just add
complexity.

data = f.readlines()
for d in data:
name, addr = line.split('\t') if line.strip() else ('', '')

> Still much to learn and the batch of downloaded PDF files should educate
> me.
>
> Regards,
>
> Rich

--
https://mail.python.org/mailman/listinfo/python-list

[Thomas Passin]

On 1/30/2024 11:25 PM, ***@gmail.com wrote:
> Thomas, on some points we may see it differently.

I'm mostly going by what the OP originally asked for back on Jan 11.
He's been too stingy with information since then to be worth spending
much time on, IMHO.

> Some formats can be done simply but are maybe better done in somewhat
> standard ways.
>
> Some of what the OP has is already tables in a database and that can
> trivially be exported into a CSV file or other formats like your TSV file
> and more. They can also import from there. As I mentioned, many spreadsheets
> and all kinds of statistical programs tend to support some formats making it
> quite flexible.
>
> Python has all kinds of functionality, such as in the pandas module, to read
> in a CSV or write it out. And once you have the data structure in memory, al
> kinds of queries and changes can be made fairly straightforwardly. As one
> example, Rich has mentioned wanting finer control in selecting who gets some
> version of the email based on concepts like market segmentation. He already
> may have info like the STATE (as in Arizona) in his database. He might at
> some point enlarge his schema so each entry is placed in one or more
> categories and thus his CSV, once imported, can do the usual tasks of
> selecting various rows and columns or doing joins or whatever.
>
> Mind you, another architecture could place quite a bit of work completely on
> the back end and he could send SQL queries to the database from python and
> get back his results into python which would then make the email messages
> and pass them on to other functionality to deliver. This would remove any
> need for files and just rely on the DB.
>
> There as as usual, too many choices and not necessarily one best answer. Of
> course if this was a major product that would be heavily used, sure, you
> could tweak and optimize. As it is, Rich is getting a chance to improve his
> python skills no matter which way he goes.
>
>
>
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
> Behalf Of Thomas Passin via Python-list
> Sent: Tuesday, January 30, 2024 10:37 PM
> To: python-***@python.org
> Subject: Re: Extract lines from file, add to new files
>
> On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
>> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>>
>>> Fine, my toy example will still be applicable. But, you know, you haven't
>>> told us enough to give you help. Do you want to replace text from values
>>> in a file? That's been covered. Do you want to send the messages using
>>> those libraries? You haven't said what you don't know how to do.
>>> Something
>>> else? What is it that you want to do that you don't know how?
>>
>> Thomas,
>>
>> For 30 years I've used a bash script using mailx to send messages to a
> list
>> of recipients. They have no salutation to personalize each one. Since I
>> want
>> to add that personalized salutation I decided to write a python script to
>> replace the bash script.
>>
>> I have collected 11 docs explaining the smtplib and email modules and
>> providing example scripts to apply them to send multiple individual
>> messages
>> with salutations and attachments.
>
> If I had a script that's been working for 30 years, I'd probably just
> use Python to do the personalizing and let the rest of the bash script
> do the rest, like it always has. The Python program would pipe or send
> the personalized messages to the rest of the bash program. Something in
> that ballpark, anyway.
>
>> Today I'm going to be reading these. They each recommend using .csv input
>> files for names and addresses. My first search is learning whether I can
>> write a single .csv file such as:
>> "name1","address1"
>> "mane2","address2"
>> which I believe will work; and by inserting at the top of the message
> block
>> Hi, {yourname}
>> the name in the .csv file will replace the bracketed place holder
> If the file contents are going to be people's names and email addresses,
> I would just tab separate them and split each line on the tab. Names
> aren't going to include tabs so that would be safe. Email addresses
> might theoretically include a tab inside a quoted name but that would be
> extremely obscure and unlikely. No need for CSV, it would just add
> complexity.
>
> data = f.readlines()
> for d in data:
> name, addr = line.split('\t') if line.strip() else ('', '')
>
>> Still much to learn and the batch of downloaded PDF files should educate
>> me.
>>
>> Regards,
>>
>> Rich
>

[Rich Shepard]

On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:

> If I had a script that's been working for 30 years, I'd probably just use
> Python to do the personalizing and let the rest of the bash script do the
> rest, like it always has. The Python program would pipe or send the
> personalized messages to the rest of the bash program. Something in that
> ballpark, anyway.

Thomas,

A bash shell script looks easier for me and more promising. Using a while
loop (one for the name file the other for the address file), and sed for
putting the name at the head of the message replacing a generic placeholder
should work with the existing for loop script.

Thanks,

Rich

[Thomas Passin]

On 1/31/2024 9:05 AM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>
>> If I had a script that's been working for 30 years, I'd probably just use
>> Python to do the personalizing and let the rest of the bash script do the
>> rest, like it always has. The Python program would pipe or send the
>> personalized messages to the rest of the bash program. Something in that
>> ballpark, anyway.
>
> Thomas,
>
> A bash shell script looks easier for me and more promising. Using a while
> loop (one for the name file the other for the address file), and sed for
> putting the name at the head of the message replacing a generic placeholder
> should work with the existing for loop script.

Sounds good. I'd still be a bit worried about the two files getting out
of sync, as others have mentioned.

[a***@gmail.com]

This discussion has circled back to where it started. It illustrates quite a
few points about how many different ways someone can do something as well as
doing it using different tools and also about how others may see aspects of
mission creep as they look for making it perfect when it need not be. I mean
the OP wants a simple personal tool and is not currently concerned with lots
of things.

The data seems to be in a data base and can be extracted from there into one
or two files mostly for convenience. Presumably, all the usual
additions/deletions/modifications are being done in the DB. If so, then as
long as his latest vision of having two synchronized files is done
consistently by making both, they are in sync.

In addition, it sounds like the original idea was a tad too simple. The OP
wanted to add a single-line or so at the head of a message that simply
interpolates a name. This could be handled with all kinds of
form-letter-merge methods as well as search and replace methods of great
generality. But, for now, the OP is asking for something truly simple and
would be happy with a solution that is slow and clunky and uses a few
utilities in a shell script on LINUX. I took a look at doing this on bash
and ran into some surprises as this would have been trivial for me in a
shell like ksh. Setting variables is now done in a subshell which was done
why? All I know is if you could simply read a file one line at a time in a
loop and set one or more variables, then it becomes easy to use something
like combining an echo and a cat of the original message to make a message
with a variable heading and feed that to the mailer. And if the storage
format was CSV, again, you could easily peel it apart using something like
cut or awk or whatever.

The main reason for trying it in python may have been frustration with doing
it as a shell script. Fair enough. But the idea of two files may have been
made in the first place by the original idea of such a script. The question
posed was sort of how to search in one and then somehow find the
corresponding part of another. The mindset of the OP may have been focused
in a direction we were not clear on. As an example, there are ways to search
file A and get back a line number of the first match and then using perhaps
another utility you ask it to scan file B and just print out that line
number and nothing else.

The current solution the OP is considering sounds like a loop that goes
through both files in parallel. This would mean when one loop finds what it
is looking for, the other delivers what is in the same location. That may
work well enough for the purpose.

But I have learned over the years that having a few well-designed tools and
ideas and using them consistently, is often a good way to get results even
if it is not as imaginative and efficient as some other. If you are allowed
to design something from top to bottom, there may be very nice solutions.
But if asked to just do one small part in a design that already feels it has
solved other aspects, then suggesting they start over is often not taken
well. LOL!

I happen to think some storage format such as CSV would be a good idea IFF
doing this in Python for some reasons I mentioned earlier. I am not in love
with CSV and use other data formats as needed and some others would be fine
as long as your toolkit included some easy and reliable way to read the
contents in and manipulate to get your results. I do programming in multiple
languages and whether I am using python or something like R, I often end up
importing the data into some data.frame format, performing a pipeline of
manipulations using it and perhaps other such structures and generating
results and optionally saving some back to files. Tons of well-tested and
comfortable tools can be used if the data is set up right so even when it
can be easily done by hand, it may not be worth the effort as the standard
tools and tricks work and work fast enough.

Rich (AKA the OP) seems satisfied with having a solution. It may not be
ideal but for his needs, ...


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Thomas Passin via Python-list
Sent: Wednesday, January 31, 2024 10:36 AM
To: python-***@python.org
Subject: Re: Extract lines from file, add to new files

On 1/31/2024 9:05 AM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>
>> If I had a script that's been working for 30 years, I'd probably just use
>> Python to do the personalizing and let the rest of the bash script do the
>> rest, like it always has. The Python program would pipe or send the
>> personalized messages to the rest of the bash program. Something in that
>> ballpark, anyway.
>
> Thomas,
>
> A bash shell script looks easier for me and more promising. Using a while
> loop (one for the name file the other for the address file), and sed for
> putting the name at the head of the message replacing a generic
placeholder
> should work with the existing for loop script.

Sounds good. I'd still be a bit worried about the two files getting out
of sync, as others have mentioned.

--
https://mail.python.org/mailman/listinfo/python-list

[a***@gmail.com]

Thomas,

I have been thinking about the concept of being stingy with information as
this is a fairly common occurrence when people ask for help. They often ask
for what they think they want while people like us keep asking why they want
that and perhaps offer guidance on how to get closer to what they NEED or a
better way.

In retrospect, Rich did give all the info he thought he needed. It boiled
down to saying that he wants to distribute data into two files in such a way
that finding an item in file A then lets him find the corresponding item in
file B. He was not worried about how to make the files or what to do with
the info afterward. He had those covered and was missing what he considered
a central piece. And, it seems he programs in multiple languages and
environments as needed and is not exactly a newbie. He just wanted a way to
implement his overall design.

We threw many solutions and ideas at him but some of us (like me) also got
frustrated as some ideas were not received due to one objection or another
that had not been mentioned earlier when it was not seen as important.

I particularly notice a disconnect some of us had. Was this supposed to be a
search that read only as much as needed to find something and stopped
reading, or a sort of filter that returned zero or more matches and went to
the end, or perhaps something that read entire files and swallowed them into
data structures in memory and then searched and found corresponding entries,
or maybe something else?

All the above approaches could work but some designs not so much. For
example, some files are too large. We, as programmers, often consciously or
unconsciously look at many factors to try to zoom in on what approaches me
might use. To be given minimal amounts of info can be frustrating. We worry
about making a silly design. But the OP may want something minimal and not
worry as long as it is fairly easy to program and works.

We could have suggested something very simple like:

Open both files A and B
In a loop get a line from each. If the line from A is a match, do something
with the current line from B.
If you are getting only one, exit the loop.

Or, if willing, we could have suggested any other file format, such as a
CSV, in which the algorithm is similar but different as in:

Open file A
Read a line in a loop
Split it in parts
If the party of the first part matches something, use the party of the
second part

Or, of course, suggest they read the entire file, into a list of lines or a
data.frame and use some tools that search all of it and produce results.

I find I personally now often lean toward the latter approach but ages ago
when memory and CPU were considerations and maybe garbage collection was not
automatic, ...


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Thomas Passin via Python-list
Sent: Wednesday, January 31, 2024 7:25 AM
To: python-***@python.org
Subject: Re: Extract lines from file, add to new files

On 1/30/2024 11:25 PM, ***@gmail.com wrote:
> Thomas, on some points we may see it differently.

I'm mostly going by what the OP originally asked for back on Jan 11.
He's been too stingy with information since then to be worth spending
much time on, IMHO.

> Some formats can be done simply but are maybe better done in somewhat
> standard ways.
>
> Some of what the OP has is already tables in a database and that can
> trivially be exported into a CSV file or other formats like your TSV file
> and more. They can also import from there. As I mentioned, many
spreadsheets
> and all kinds of statistical programs tend to support some formats making
it
> quite flexible.
>
> Python has all kinds of functionality, such as in the pandas module, to
read
> in a CSV or write it out. And once you have the data structure in memory,
al
> kinds of queries and changes can be made fairly straightforwardly. As one
> example, Rich has mentioned wanting finer control in selecting who gets
some
> version of the email based on concepts like market segmentation. He
already
> may have info like the STATE (as in Arizona) in his database. He might at
> some point enlarge his schema so each entry is placed in one or more
> categories and thus his CSV, once imported, can do the usual tasks of
> selecting various rows and columns or doing joins or whatever.
>
> Mind you, another architecture could place quite a bit of work completely
on
> the back end and he could send SQL queries to the database from python and
> get back his results into python which would then make the email messages
> and pass them on to other functionality to deliver. This would remove any
> need for files and just rely on the DB.
>
> There as as usual, too many choices and not necessarily one best answer.
Of
> course if this was a major product that would be heavily used, sure, you
> could tweak and optimize. As it is, Rich is getting a chance to improve
his
> python skills no matter which way he goes.
>
>
>
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=***@python.org>
On
> Behalf Of Thomas Passin via Python-list
> Sent: Tuesday, January 30, 2024 10:37 PM
> To: python-***@python.org
> Subject: Re: Extract lines from file, add to new files
>
> On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
>> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>>
>>> Fine, my toy example will still be applicable. But, you know, you
haven't
>>> told us enough to give you help. Do you want to replace text from values
>>> in a file? That's been covered. Do you want to send the messages using
>>> those libraries? You haven't said what you don't know how to do.
>>> Something
>>> else? What is it that you want to do that you don't know how?
>>
>> Thomas,
>>
>> For 30 years I've used a bash script using mailx to send messages to a
> list
>> of recipients. They have no salutation to personalize each one. Since I
>> want
>> to add that personalized salutation I decided to write a python script to
>> replace the bash script.
>>
>> I have collected 11 docs explaining the smtplib and email modules and
>> providing example scripts to apply them to send multiple individual
>> messages
>> with salutations and attachments.
>
> If I had a script that's been working for 30 years, I'd probably just
> use Python to do the personalizing and let the rest of the bash script
> do the rest, like it always has. The Python program would pipe or send
> the personalized messages to the rest of the bash program. Something in
> that ballpark, anyway.
>
>> Today I'm going to be reading these. They each recommend using .csv input
>> files for names and addresses. My first search is learning whether I can
>> write a single .csv file such as:
>> "name1","address1"
>> "mane2","address2"
>> which I believe will work; and by inserting at the top of the message
> block
>> Hi, {yourname}
>> the name in the .csv file will replace the bracketed place holder
> If the file contents are going to be people's names and email addresses,
> I would just tab separate them and split each line on the tab. Names
> aren't going to include tabs so that would be safe. Email addresses
> might theoretically include a tab inside a quoted name but that would be
> extremely obscure and unlikely. No need for CSV, it would just add
> complexity.
>
> data = f.readlines()
> for d in data:
> name, addr = line.split('\t') if line.strip() else ('', '')
>
>> Still much to learn and the batch of downloaded PDF files should educate
>> me.
>>
>> Regards,
>>
>> Rich
>

--
https://mail.python.org/mailman/listinfo/python-list

[Thomas Passin]

In my view this whole thread became murky and complicated because the OP
did not write down the requirements for the program. Requirements are
needed to communicate with other people. An individual may not need to
actually write down the requirements - depending on their complexity -
but they always exist even if only vaguely in a person's mind. The
requirements may include what tools or languages the person wants to use
and why.

If you are asking for help, you need to communicate the requirements to
the people you are asking for help from.

The OP may have thought the original post(s) contained enough of the
requirements but as we know by now, they didn't.

The person asking for help may not realize they don't know enough to
write down all the requirements; an effort to do so may bring that lack
to visibility.

Mailing lists like these have a drawback that it's hard to impossible
for someone not involved in a thread to learn anything general from it.
We can write over and over again to please state clearly what you want
to do and where the sticking points are, but newcomers post new
questions without ever reading these pleas. Then good-hearted people
who want to be helpful end up spending a lot of time trying to guess
what is actually being asked for, and maybe never find out with enough
clarity. Others take a guess and then spend time working up a solution
that may or may not be on target.

So please! before posting a request for help, write down the
requirements as best you can figure them out, and then make sure that
they are expressed such that the readers can understand.

On 2/3/2024 11:33 AM, ***@gmail.com wrote:
> Thomas,
>
> I have been thinking about the concept of being stingy with information as
> this is a fairly common occurrence when people ask for help. They often ask
> for what they think they want while people like us keep asking why they want
> that and perhaps offer guidance on how to get closer to what they NEED or a
> better way.
>
> In retrospect, Rich did give all the info he thought he needed. It boiled
> down to saying that he wants to distribute data into two files in such a way
> that finding an item in file A then lets him find the corresponding item in
> file B. He was not worried about how to make the files or what to do with
> the info afterward. He had those covered and was missing what he considered
> a central piece. And, it seems he programs in multiple languages and
> environments as needed and is not exactly a newbie. He just wanted a way to
> implement his overall design.
>
> We threw many solutions and ideas at him but some of us (like me) also got
> frustrated as some ideas were not received due to one objection or another
> that had not been mentioned earlier when it was not seen as important.
>
> I particularly notice a disconnect some of us had. Was this supposed to be a
> search that read only as much as needed to find something and stopped
> reading, or a sort of filter that returned zero or more matches and went to
> the end, or perhaps something that read entire files and swallowed them into
> data structures in memory and then searched and found corresponding entries,
> or maybe something else?
>
> All the above approaches could work but some designs not so much. For
> example, some files are too large. We, as programmers, often consciously or
> unconsciously look at many factors to try to zoom in on what approaches me
> might use. To be given minimal amounts of info can be frustrating. We worry
> about making a silly design. But the OP may want something minimal and not
> worry as long as it is fairly easy to program and works.
>
> We could have suggested something very simple like:
>
> Open both files A and B
> In a loop get a line from each. If the line from A is a match, do something
> with the current line from B.
> If you are getting only one, exit the loop.
>
> Or, if willing, we could have suggested any other file format, such as a
> CSV, in which the algorithm is similar but different as in:
>
> Open file A
> Read a line in a loop
> Split it in parts
> If the party of the first part matches something, use the party of the
> second part
>
> Or, of course, suggest they read the entire file, into a list of lines or a
> data.frame and use some tools that search all of it and produce results.
>
> I find I personally now often lean toward the latter approach but ages ago
> when memory and CPU were considerations and maybe garbage collection was not
> automatic, ...
>
>
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
> Behalf Of Thomas Passin via Python-list
> Sent: Wednesday, January 31, 2024 7:25 AM
> To: python-***@python.org
> Subject: Re: Extract lines from file, add to new files
>
> On 1/30/2024 11:25 PM, ***@gmail.com wrote:
>> Thomas, on some points we may see it differently.
>
> I'm mostly going by what the OP originally asked for back on Jan 11.
> He's been too stingy with information since then to be worth spending
> much time on, IMHO.
>
>> Some formats can be done simply but are maybe better done in somewhat
>> standard ways.
>>
>> Some of what the OP has is already tables in a database and that can
>> trivially be exported into a CSV file or other formats like your TSV file
>> and more. They can also import from there. As I mentioned, many
> spreadsheets
>> and all kinds of statistical programs tend to support some formats making
> it
>> quite flexible.
>>
>> Python has all kinds of functionality, such as in the pandas module, to
> read
>> in a CSV or write it out. And once you have the data structure in memory,
> al
>> kinds of queries and changes can be made fairly straightforwardly. As one
>> example, Rich has mentioned wanting finer control in selecting who gets
> some
>> version of the email based on concepts like market segmentation. He
> already
>> may have info like the STATE (as in Arizona) in his database. He might at
>> some point enlarge his schema so each entry is placed in one or more
>> categories and thus his CSV, once imported, can do the usual tasks of
>> selecting various rows and columns or doing joins or whatever.
>>
>> Mind you, another architecture could place quite a bit of work completely
> on
>> the back end and he could send SQL queries to the database from python and
>> get back his results into python which would then make the email messages
>> and pass them on to other functionality to deliver. This would remove any
>> need for files and just rely on the DB.
>>
>> There as as usual, too many choices and not necessarily one best answer.
> Of
>> course if this was a major product that would be heavily used, sure, you
>> could tweak and optimize. As it is, Rich is getting a chance to improve
> his
>> python skills no matter which way he goes.
>>
>>
>>
>> -----Original Message-----
>> From: Python-list <python-list-bounces+avi.e.gross=***@python.org>
> On
>> Behalf Of Thomas Passin via Python-list
>> Sent: Tuesday, January 30, 2024 10:37 PM
>> To: python-***@python.org
>> Subject: Re: Extract lines from file, add to new files
>>
>> On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
>>> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>>>
>>>> Fine, my toy example will still be applicable. But, you know, you
> haven't
>>>> told us enough to give you help. Do you want to replace text from values
>>>> in a file? That's been covered. Do you want to send the messages using
>>>> those libraries? You haven't said what you don't know how to do.
>>>> Something
>>>> else? What is it that you want to do that you don't know how?
>>>
>>> Thomas,
>>>
>>> For 30 years I've used a bash script using mailx to send messages to a
>> list
>>> of recipients. They have no salutation to personalize each one. Since I
>>> want
>>> to add that personalized salutation I decided to write a python script to
>>> replace the bash script.
>>>
>>> I have collected 11 docs explaining the smtplib and email modules and
>>> providing example scripts to apply them to send multiple individual
>>> messages
>>> with salutations and attachments.
>>
>> If I had a script that's been working for 30 years, I'd probably just
>> use Python to do the personalizing and let the rest of the bash script
>> do the rest, like it always has. The Python program would pipe or send
>> the personalized messages to the rest of the bash program. Something in
>> that ballpark, anyway.
>>
>>> Today I'm going to be reading these. They each recommend using .csv input
>>> files for names and addresses. My first search is learning whether I can
>>> write a single .csv file such as:
>>> "name1","address1"
>>> "mane2","address2"
>>> which I believe will work; and by inserting at the top of the message
>> block
>>> Hi, {yourname}
>>> the name in the .csv file will replace the bracketed place holder
>> If the file contents are going to be people's names and email addresses,
>> I would just tab separate them and split each line on the tab. Names
>> aren't going to include tabs so that would be safe. Email addresses
>> might theoretically include a tab inside a quoted name but that would be
>> extremely obscure and unlikely. No need for CSV, it would just add
>> complexity.
>>
>> data = f.readlines()
>> for d in data:
>> name, addr = line.split('\t') if line.strip() else ('', '')
>>
>>> Still much to learn and the batch of downloaded PDF files should educate
>>> me.
>>>
>>> Regards,
>>>
>>> Rich
>>
>

[Mats Wichmann]

On 2/3/24 10:58, Thomas Passin via Python-list wrote:
> In my view this whole thread became murky and complicated because the OP
> did not write down the requirements for the program.  Requirements are
> needed to communicate with other people.  An individual may not need to
> actually write down the requirements - depending on their complexity -
> but they always exist even if only vaguely in a person's mind.  The
> requirements may include what tools or languages the person wants to use
> and why.
>
> If you are asking for help, you need to communicate the requirements to
> the people you are asking for help from.
>
> The OP may have thought the original post(s) contained enough of the
> requirements but as we know by now, they didn't.
>
> The person asking for help may not realize they don't know enough to
> write down all the requirements; an effort to do so may bring that lack
> to visibility.
>
> Mailing lists like these have a drawback that it's hard to impossible
> for someone not involved in a thread to learn anything general from it.
> We can write over and over again to please state clearly what you want
> to do and where the sticking points are, but newcomers post new
> questions without ever reading these pleas.  Then good-hearted people
> who want to be helpful end up spending a lot of time trying to guess
> what is actually being asked for, and maybe never find out with enough
> clarity.  Others take a guess and then spend time working up a solution
> that may or may not be on target.
>
> So please! before posting a request for help, write down the
> requirements as best you can figure them out, and then make sure that
> they are expressed such that the readers can understand.

Indeed. I've occasionally practised the following technique (in some
form) over the years without knowing it had grown a name, and wikipedia
page to go with it. It may be handy to use to help come up with a
clearer explanation before sending off a post to a mailing list or other
static medium, because of the inevitable delays in going back and forth.
Interactive formus like the Python Discord have a bit of an advantage in
that you can try to tease out the intent more quickly. But as you
say... a newcomer won't know this.

https://en.wikipedia.org/wiki/Rubber_duck_debugging

[a***@gmail.com]

We substantially agree with that, Thomas. In the best of all possible
worlds, someone who gets stuck will sit down and try to carefully spell out
things in ways like you mention and, incidentally, may often catch the error
or figure out how to do it and not even send in a request! LOL!

I think a main thing that the OP and others can do is to not just be
abstract but supply a small example including what output they expect and
perhaps what they did not receive properly along with error messages.

Rich had not tried doing what he wanted in python, yet. I don't know if he
did any other parts yet. I think he was still in a somewhat abstract design
state and hoping for someone to push him at something like a resource to
continue and he did accept suggestions on what now seem somewhat random
things to read as people guessed.

In a sense, I and others can take some blame for the way we widened the
problem while trying to look at it our own way.

But looking back, I think Rich asked close to what he wanted. An example
might have helped such as:

'I have information about multiple clients including an email address and a
human name such as:

***@domain.com Sally
***@bark.com Barky

I want to save the info in a file or maybe two in such a way that when I
write a program and ask it to send an email to "***@bark.com" then it finds
an entry that matches the email address and then uses that to find the
matching name and sends a specified message with a specific salutation in
front like "Dear Barky,".

I was thinking of having two files with one having email address after
another and the other having the corresponding names. I want to use python
to search one file and then return the name in the other. Are there other
and perhaps better ways commonly used to associate one keyword with a
value?'

Back to me. The above is not polished but might still have engendered a
discussion such as how to keep the files synchronized. Yes, the OP could
have added endless clauses saying that they are not asking how to create the
data files or keep them synchronized but just how to match them. The reply
in this case possibly could have suggested they count the lines they have
read until the match and, assuming no blank lines are used, read a second
file till the nth line. We might have been done quickly and THEN had a long
discussion about other ways!

I have participated, like you, in another forum designed for tutoring and I
think the rules and expectations there may be a bit different. Over here, it
is fairer to expect people to take a bit of time and ask clearer questions.

We live and we learn and then Alzheimer's ...



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On
Behalf Of Thomas Passin via Python-list
Sent: Saturday, February 3, 2024 12:59 PM
To: python-***@python.org
Subject: Re: Extract lines from file, add to new files

In my view this whole thread became murky and complicated because the OP
did not write down the requirements for the program. Requirements are
needed to communicate with other people. An individual may not need to
actually write down the requirements - depending on their complexity -
but they always exist even if only vaguely in a person's mind. The
requirements may include what tools or languages the person wants to use
and why.

If you are asking for help, you need to communicate the requirements to
the people you are asking for help from.

The OP may have thought the original post(s) contained enough of the
requirements but as we know by now, they didn't.

The person asking for help may not realize they don't know enough to
write down all the requirements; an effort to do so may bring that lack
to visibility.

Mailing lists like these have a drawback that it's hard to impossible
for someone not involved in a thread to learn anything general from it.
We can write over and over again to please state clearly what you want
to do and where the sticking points are, but newcomers post new
questions without ever reading these pleas. Then good-hearted people
who want to be helpful end up spending a lot of time trying to guess
what is actually being asked for, and maybe never find out with enough
clarity. Others take a guess and then spend time working up a solution
that may or may not be on target.

So please! before posting a request for help, write down the
requirements as best you can figure them out, and then make sure that
they are expressed such that the readers can understand.

On 2/3/2024 11:33 AM, ***@gmail.com wrote:
> Thomas,
>
> I have been thinking about the concept of being stingy with information as
> this is a fairly common occurrence when people ask for help. They often
ask
> for what they think they want while people like us keep asking why they
want
> that and perhaps offer guidance on how to get closer to what they NEED or
a
> better way.
>
> In retrospect, Rich did give all the info he thought he needed. It boiled
> down to saying that he wants to distribute data into two files in such a
way
> that finding an item in file A then lets him find the corresponding item
in
> file B. He was not worried about how to make the files or what to do with
> the info afterward. He had those covered and was missing what he
considered
> a central piece. And, it seems he programs in multiple languages and
> environments as needed and is not exactly a newbie. He just wanted a way
to
> implement his overall design.
>
> We threw many solutions and ideas at him but some of us (like me) also got
> frustrated as some ideas were not received due to one objection or another
> that had not been mentioned earlier when it was not seen as important.
>
> I particularly notice a disconnect some of us had. Was this supposed to be
a
> search that read only as much as needed to find something and stopped
> reading, or a sort of filter that returned zero or more matches and went
to
> the end, or perhaps something that read entire files and swallowed them
into
> data structures in memory and then searched and found corresponding
entries,
> or maybe something else?
>
> All the above approaches could work but some designs not so much. For
> example, some files are too large. We, as programmers, often consciously
or
> unconsciously look at many factors to try to zoom in on what approaches me
> might use. To be given minimal amounts of info can be frustrating. We
worry
> about making a silly design. But the OP may want something minimal and not
> worry as long as it is fairly easy to program and works.
>
> We could have suggested something very simple like:
>
> Open both files A and B
> In a loop get a line from each. If the line from A is a match, do
something
> with the current line from B.
> If you are getting only one, exit the loop.
>
> Or, if willing, we could have suggested any other file format, such as a
> CSV, in which the algorithm is similar but different as in:
>
> Open file A
> Read a line in a loop
> Split it in parts
> If the party of the first part matches something, use the party of the
> second part
>
> Or, of course, suggest they read the entire file, into a list of lines or
a
> data.frame and use some tools that search all of it and produce results.
>
> I find I personally now often lean toward the latter approach but ages ago
> when memory and CPU were considerations and maybe garbage collection was
not
> automatic, ...
>
>
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=***@python.org>
On
> Behalf Of Thomas Passin via Python-list
> Sent: Wednesday, January 31, 2024 7:25 AM
> To: python-***@python.org
> Subject: Re: Extract lines from file, add to new files
>
> On 1/30/2024 11:25 PM, ***@gmail.com wrote:
>> Thomas, on some points we may see it differently.
>
> I'm mostly going by what the OP originally asked for back on Jan 11.
> He's been too stingy with information since then to be worth spending
> much time on, IMHO.
>
>> Some formats can be done simply but are maybe better done in somewhat
>> standard ways.
>>
>> Some of what the OP has is already tables in a database and that can
>> trivially be exported into a CSV file or other formats like your TSV file
>> and more. They can also import from there. As I mentioned, many
> spreadsheets
>> and all kinds of statistical programs tend to support some formats making
> it
>> quite flexible.
>>
>> Python has all kinds of functionality, such as in the pandas module, to
> read
>> in a CSV or write it out. And once you have the data structure in memory,
> al
>> kinds of queries and changes can be made fairly straightforwardly. As one
>> example, Rich has mentioned wanting finer control in selecting who gets
> some
>> version of the email based on concepts like market segmentation. He
> already
>> may have info like the STATE (as in Arizona) in his database. He might at
>> some point enlarge his schema so each entry is placed in one or more
>> categories and thus his CSV, once imported, can do the usual tasks of
>> selecting various rows and columns or doing joins or whatever.
>>
>> Mind you, another architecture could place quite a bit of work completely
> on
>> the back end and he could send SQL queries to the database from python
and
>> get back his results into python which would then make the email messages
>> and pass them on to other functionality to deliver. This would remove any
>> need for files and just rely on the DB.
>>
>> There as as usual, too many choices and not necessarily one best answer.
> Of
>> course if this was a major product that would be heavily used, sure, you
>> could tweak and optimize. As it is, Rich is getting a chance to improve
> his
>> python skills no matter which way he goes.
>>
>>
>>
>> -----Original Message-----
>> From: Python-list <python-list-bounces+avi.e.gross=***@python.org>
> On
>> Behalf Of Thomas Passin via Python-list
>> Sent: Tuesday, January 30, 2024 10:37 PM
>> To: python-***@python.org
>> Subject: Re: Extract lines from file, add to new files
>>
>> On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
>>> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>>>
>>>> Fine, my toy example will still be applicable. But, you know, you
> haven't
>>>> told us enough to give you help. Do you want to replace text from
values
>>>> in a file? That's been covered. Do you want to send the messages using
>>>> those libraries? You haven't said what you don't know how to do.
>>>> Something
>>>> else? What is it that you want to do that you don't know how?
>>>
>>> Thomas,
>>>
>>> For 30 years I've used a bash script using mailx to send messages to a
>> list
>>> of recipients. They have no salutation to personalize each one. Since I
>>> want
>>> to add that personalized salutation I decided to write a python script
to
>>> replace the bash script.
>>>
>>> I have collected 11 docs explaining the smtplib and email modules and
>>> providing example scripts to apply them to send multiple individual
>>> messages
>>> with salutations and attachments.
>>
>> If I had a script that's been working for 30 years, I'd probably just
>> use Python to do the personalizing and let the rest of the bash script
>> do the rest, like it always has. The Python program would pipe or send
>> the personalized messages to the rest of the bash program. Something in
>> that ballpark, anyway.
>>
>>> Today I'm going to be reading these. They each recommend using .csv
input
>>> files for names and addresses. My first search is learning whether I can
>>> write a single .csv file such as:
>>> "name1","address1"
>>> "mane2","address2"
>>> which I believe will work; and by inserting at the top of the message
>> block
>>> Hi, {yourname}
>>> the name in the .csv file will replace the bracketed place holder
>> If the file contents are going to be people's names and email addresses,
>> I would just tab separate them and split each line on the tab. Names
>> aren't going to include tabs so that would be safe. Email addresses
>> might theoretically include a tab inside a quoted name but that would be
>> extremely obscure and unlikely. No need for CSV, it would just add
>> complexity.
>>
>> data = f.readlines()
>> for d in data:
>> name, addr = line.split('\t') if line.strip() else ('', '')
>>
>>> Still much to learn and the batch of downloaded PDF files should educate
>>> me.
>>>
>>> Regards,
>>>
>>> Rich
>>
>

--
https://mail.python.org/mailman/listinfo/python-list

[dn]

Every trainer, in any field, has to deal with these problems - all the
time, and over-and-over.


On 4/02/24 06:58, Thomas Passin via Python-list wrote:
> In my view this whole thread became murky and complicated because the OP
> did not write down the requirements for the program.  Requirements are
> needed to communicate with other people.  An individual may not need to
> actually write down the requirements - depending on their complexity -
> but they always exist even if only vaguely in a person's mind.  The
> requirements may include what tools or languages the person wants to use
> and why.
>
> If you are asking for help, you need to communicate the requirements to
> the people you are asking for help from.
>
> The OP may have thought the original post(s) contained enough of the
> requirements but as we know by now, they didn't.

There is another possible interpretation in such situations (not
necessarily this one): that the person is fixated on a particular
solution (and unable/unwilling to adjust his/her thinking to consider
more widely).

Thus, the question is not: 'here's an entire problem, how can it be
solved', but more: 'I have a solution, and want help to implement it
(and only it) just-so'.


The latter is an interesting psychology:

1
an experienced person who is trying to translate from one tool to
another (Python), but discovers that a word-for-word solution is
difficult because of the artificial-constraints they've placed on the
situation.

2
a beginner who doesn't know what (s)he doesn't know and comes-up with an
idea, but fails to appreciate that there is likely more than one path to
the goal.


> The person asking for help may not realize they don't know enough to
> write down all the requirements; an effort to do so may bring that lack
> to visibility.

In the case of 'Beginners' this should probably be taken-as-read!

Which is why we will always find ourselves asking questions or 'please
give more information'...


However, there are other reasons, eg corporate concerns or personality;
why people don't want to give more information. The former is reasonable
(have suffered from same myself). The latter may reveal that the person
is 'difficult to deal with'...


> Mailing lists like these have a drawback that it's hard to impossible
> for someone not involved in a thread to learn anything general from it.
> We can write over and over again to please state clearly what you want
> to do and where the sticking points are, but newcomers post new
> questions without ever reading these pleas.  Then good-hearted people
> who want to be helpful end up spending a lot of time trying to guess
> what is actually being asked for, and maybe never find out with enough
> clarity.  Others take a guess and then spend time working up a solution
> that may or may not be on target.
>
> So please! before posting a request for help, write down the
> requirements as best you can figure them out, and then make sure that
> they are expressed such that the readers can understand.

Unfortunately, if the person doesn't understand the problem (leave-aside
any ideas of solution), then (s)he will not be able to clearly
communicate same to us, in any way, shape, or form...

Which brings one to the question: if a person cannot express the problem
clearly and completely, is (s)he suited to development work? If the
problem is not understood, could 'the solution' ever be more than an
exercise in hope?
(prototyping and experimentation aside)


Yes, it is frustrating to invest time and effort in helping someone,
only for same to disappear 'into a black hole'. The lack of response
seems to indicate a lack of respect or appreciation. Is this perhaps
part of today's "consumer" life-style, where so few are contributors or
creators?


On the other side of that coin: do the people who make assumptions and
(kindly) blaze-ahead with 'a solution', actually help the conversation?
If the assumptions are correct, yes! What if they are not?


...and don't get me started on folk who want us to do their
training-assignments or build some application, for them!


As a slight aside: on one training-course DiscussionList/BulletinBoard
set-up, if a trainee asked a question without a descriptive
title/SubjectLine, eg "Python not working" or "Urgent: please help"; I
asked them to re-post with a title that would help others in a similar
situation find the topic - and closed the original thread.

Some found it "brutal" - probably skewing towards those who felt
"Urgent" because they'd left things too close to deadline. Others joined
the (later) thread because they could identify the topic and realise
their interest in learning or contributing to the conversation...


Time pressures lead to a multitude of evils!


There's a quotation that goes something like "the poor will be with your
always"?
(?possibly Biblical)

Whether we (here) are talking about 'poor' manners, 'poor'
understanding, 'poor' communication skills, 'poor' Python knowledge, or
whatever; isn't such one of the rationales for this DiscussionList?

That said, we're all volunteering our (valuable) time!
--
Regards,
=dn

[Thomas Passin]

On 2/3/2024 5:02 PM, dn via Python-list wrote:
> Every trainer, in any field, has to deal with these problems - all the
> time, and over-and-over.
>
>
> On 4/02/24 06:58, Thomas Passin via Python-list wrote:
>> In my view this whole thread became murky and complicated because the
>> OP did not write down the requirements for the program.  Requirements
>> are needed to communicate with other people.  An individual may not
>> need to actually write down the requirements - depending on their
>> complexity - but they always exist even if only vaguely in a person's
>> mind.  The requirements may include what tools or languages the person
>> wants to use and why.
>>
>> If you are asking for help, you need to communicate the requirements
>> to the people you are asking for help from.
>>
>> The OP may have thought the original post(s) contained enough of the
>> requirements but as we know by now, they didn't.
>
> There is another possible interpretation in such situations (not
> necessarily this one): that the person is fixated on a particular
> solution (and unable/unwilling to adjust his/her thinking to consider
> more widely).
>
> Thus, the question is not: 'here's an entire problem, how can it be
> solved', but more: 'I have a solution, and want help to implement it
> (and only it) just-so'.
>
>
> The latter is an interesting psychology:
>
> 1
> an experienced person who is trying to translate from one tool to
> another (Python), but discovers that a word-for-word solution is
> difficult because of the artificial-constraints they've placed on the
> situation.
>
> 2
> a beginner who doesn't know what (s)he doesn't know and comes-up with an
> idea, but fails to appreciate that there is likely more than one path to
> the goal.
>
>
>> The person asking for help may not realize they don't know enough to
>> write down all the requirements; an effort to do so may bring that
>> lack to visibility.
>
> In the case of 'Beginners' this should probably be taken-as-read!
>
> Which is why we will always find ourselves asking questions or 'please
> give more information'...
>
>
> However, there are other reasons, eg corporate concerns or personality;
> why people don't want to give more information. The former is reasonable
> (have suffered from same myself). The latter may reveal that the person
> is 'difficult to deal with'...
>
>
>> Mailing lists like these have a drawback that it's hard to impossible
>> for someone not involved in a thread to learn anything general from
>> it. We can write over and over again to please state clearly what you
>> want to do and where the sticking points are, but newcomers post new
>> questions without ever reading these pleas.  Then good-hearted people
>> who want to be helpful end up spending a lot of time trying to guess
>> what is actually being asked for, and maybe never find out with enough
>> clarity.  Others take a guess and then spend time working up a
>> solution that may or may not be on target.
>>
>> So please! before posting a request for help, write down the
>> requirements as best you can figure them out, and then make sure that
>> they are expressed such that the readers can understand.
>
> Unfortunately, if the person doesn't understand the problem (leave-aside
> any ideas of solution), then (s)he will not be able to clearly
> communicate same to us, in any way, shape, or form...
>
> Which brings one to the question: if a person cannot express the problem
> clearly and completely, is (s)he suited to development work? If the
> problem is not understood, could 'the solution' ever be more than an
> exercise in hope?
> (prototyping and experimentation aside)

Pairs programming can be fun and productive, if you are lucky to have
the right person to work with. I've had one person like that over the
years.

> Yes, it is frustrating to invest time and effort in helping someone,
> only for same to disappear 'into a black hole'. The lack of response
> seems to indicate a lack of respect or appreciation. Is this perhaps
> part of today's "consumer" life-style, where so few are contributors or
> creators?
>
>
> On the other side of that coin: do the people who make assumptions and
> (kindly) blaze-ahead with 'a solution', actually help the conversation?
> If the assumptions are correct, yes! What if they are not?
>
>
> ...and don't get me started on folk who want us to do their
> training-assignments or build some application, for them!
>
>
> As a slight aside: on one training-course DiscussionList/BulletinBoard
> set-up, if a trainee asked a question without a descriptive
> title/SubjectLine, eg "Python not working" or "Urgent: please help"; I
> asked them to re-post with a title that would help others in a similar
> situation find the topic - and closed the original thread.
>
> Some found it "brutal" - probably skewing towards those who felt
> "Urgent" because they'd left things too close to deadline. Others joined
> the (later) thread because they could identify the topic and realise
> their interest in learning or contributing to the conversation...
>
>
> Time pressures lead to a multitude of evils!
>
>
> There's a quotation that goes something like "the poor will be with your
> always"?
> (?possibly Biblical)
>
> Whether we (here) are talking about 'poor' manners, 'poor'
> understanding, 'poor' communication skills, 'poor' Python knowledge, or
> whatever; isn't such one of the rationales for this DiscussionList?
>
> That said, we're all volunteering our (valuable) time!

[a***@gmail.com]

Dave,

You and I have had some experience in teaching or tutoring others and I think it fair to say our motivation is closer to teaching someone how they can fish for themselves rather than just handing them a fully-cooked fish.

My favorite kinds of questions, thus, include someone who explains what they are trying to do and shows some code along with indications of what it produced (including error messages) and what it should produce. Then the question should not be a request to just FIX THIS or WRITE IT FOR ME but asking if someone can show what they did wrong with some idea where it went wrong.

This may not be so common but it allows faster and easier help.

But others out there probably can be helped including people who want suggestions of approaches to try.

We have discussed some issues here and elsewhere where some purists tend to want to stick with close to basic aspects of the language. In some R communities, some get upset if someone asks a question (or others supply an answer) using non-official add-ons called the tidyverse and I suspect some here don't really like discussions that focus mainly on numpy/pandas or so many other modules. Yet, the reality is, that except for students who are supposed to be learning the basics of the language and show solutions compatible with what is being taught in class, others can and should be open to simple solutions using non-standard tools that are widely available and vetted. If a student is asked to use some method to calculate a value like pi, then suggesting they use math.pi is not useful. But if the assignment is to do something in trigonometry, it seems a good thing to use as a part of a larger solution rather than either embedding a constant to N decimal places or calculating it on the fly.

I think people like us who volunteer to reply should consider our choices too. I think it is fair to reply, as I saw on the tutor forum, that the code shown uses a method that is not the way the replier would do it but nonetheless offer some thoughts on particular python coding errors. I am talking about someone who wanted to implement a binary tree using a numpy array or something with minimal explanation and demanded a fix. They never supplied enough info and I think the few who replied backed away. As you mention, they seemed resistant to starting over and using data structures that are perhaps more amenable, and in any case, the code shown seemed mostly focused on how to make random and non-redundant names.

I will end with a comment. I have heard of interview techniques for a job where they deliberately supply a problem in which the goal is not so much to be easy to solve in front of them in real time but to watch how the person looking for a job responds to the uncertainties and asks follow-up questions or verbalizes things like, if it is like this, I might use this technique but if you also need that then ...

So, I shudder to think what happens if someone being interviewed turns around and asks us and further confuses things with changes to make it harder to recognize they are asking for outside help. The answer expected may well be to NOT use say the older versions of PASCAL to do something but switch to something better suited (and for that matter available.) I would not want to program the DES encryption/decryption method in Pascal again! And these days, it seems much better to just find a module or package that meets such needs.

Avi



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=***@python.org> On Behalf Of dn via Python-list
Sent: Saturday, February 3, 2024 5:02 PM
To: python-***@python.org
Subject: Re: Extract lines from file, add to new files

Every trainer, in any field, has to deal with these problems - all the
time, and over-and-over.


On 4/02/24 06:58, Thomas Passin via Python-list wrote:
> In my view this whole thread became murky and complicated because the OP
> did not write down the requirements for the program. Requirements are
> needed to communicate with other people. An individual may not need to
> actually write down the requirements - depending on their complexity -
> but they always exist even if only vaguely in a person's mind. The
> requirements may include what tools or languages the person wants to use
> and why.
>
> If you are asking for help, you need to communicate the requirements to
> the people you are asking for help from.
>
> The OP may have thought the original post(s) contained enough of the
> requirements but as we know by now, they didn't.

There is another possible interpretation in such situations (not
necessarily this one): that the person is fixated on a particular
solution (and unable/unwilling to adjust his/her thinking to consider
more widely).

Thus, the question is not: 'here's an entire problem, how can it be
solved', but more: 'I have a solution, and want help to implement it
(and only it) just-so'.


The latter is an interesting psychology:

1
an experienced person who is trying to translate from one tool to
another (Python), but discovers that a word-for-word solution is
difficult because of the artificial-constraints they've placed on the
situation.

2
a beginner who doesn't know what (s)he doesn't know and comes-up with an
idea, but fails to appreciate that there is likely more than one path to
the goal.


> The person asking for help may not realize they don't know enough to
> write down all the requirements; an effort to do so may bring that lack
> to visibility.

In the case of 'Beginners' this should probably be taken-as-read!

Which is why we will always find ourselves asking questions or 'please
give more information'...


However, there are other reasons, eg corporate concerns or personality;
why people don't want to give more information. The former is reasonable
(have suffered from same myself). The latter may reveal that the person
is 'difficult to deal with'...


> Mailing lists like these have a drawback that it's hard to impossible
> for someone not involved in a thread to learn anything general from it.
> We can write over and over again to please state clearly what you want
> to do and where the sticking points are, but newcomers post new
> questions without ever reading these pleas. Then good-hearted people
> who want to be helpful end up spending a lot of time trying to guess
> what is actually being asked for, and maybe never find out with enough
> clarity. Others take a guess and then spend time working up a solution
> that may or may not be on target.
>
> So please! before posting a request for help, write down the
> requirements as best you can figure them out, and then make sure that
> they are expressed such that the readers can understand.

Unfortunately, if the person doesn't understand the problem (leave-aside
any ideas of solution), then (s)he will not be able to clearly
communicate same to us, in any way, shape, or form...

Which brings one to the question: if a person cannot express the problem
clearly and completely, is (s)he suited to development work? If the
problem is not understood, could 'the solution' ever be more than an
exercise in hope?
(prototyping and experimentation aside)


Yes, it is frustrating to invest time and effort in helping someone,
only for same to disappear 'into a black hole'. The lack of response
seems to indicate a lack of respect or appreciation. Is this perhaps
part of today's "consumer" life-style, where so few are contributors or
creators?


On the other side of that coin: do the people who make assumptions and
(kindly) blaze-ahead with 'a solution', actually help the conversation?
If the assumptions are correct, yes! What if they are not?


...and don't get me started on folk who want us to do their
training-assignments or build some application, for them!


As a slight aside: on one training-course DiscussionList/BulletinBoard
set-up, if a trainee asked a question without a descriptive
title/SubjectLine, eg "Python not working" or "Urgent: please help"; I
asked them to re-post with a title that would help others in a similar
situation find the topic - and closed the original thread.

Some found it "brutal" - probably skewing towards those who felt
"Urgent" because they'd left things too close to deadline. Others joined
the (later) thread because they could identify the topic and realise
their interest in learning or contributing to the conversation...


Time pressures lead to a multitude of evils!


There's a quotation that goes something like "the poor will be with your
always"?
(?possibly Biblical)

Whether we (here) are talking about 'poor' manners, 'poor'
understanding, 'poor' communication skills, 'poor' Python knowledge, or
whatever; isn't such one of the rationales for this DiscussionList?

That said, we're all volunteering our (valuable) time!
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

[dn]

On 4/02/24 13:20, ***@gmail.com wrote:
> Dave,
>
> You and I have had some experience in teaching or tutoring others and I think it fair to say our motivation is closer to teaching someone how they can fish for themselves rather than just handing them a fully-cooked fish.

Which may push the debate 'up' a level, in that there are two pertinent
Python Discussion Lists (amongst the many): Tutor which is expressly for
learners (and tutors), and this one which is to discuss Python.
Accordingly, one might suggest that people 'here' are looking for a
direct answer - the fish (having tried to fish for themselves), but
learners (seeking to learn to fish) should be asking elsewhere.

This would sort-out the type/level of questions that OPs may have. As
well as indicating an appropriate approach to 'answers'.

However, there's no rule which says one has to ask in one place or the
other (nor am I suggesting such, although...). Then again, might the
lack of forethought evident in some questions and the forum-used,
indicate a type of person who wouldn't investigate to see which is the
best place for his/her enquiries anyway?

Tutor List: https://mail.python.org/mailman/listinfo/tutor
Lists Overview: https://mail.python.org/mailman/listinfo


> My favorite kinds of questions, thus, include someone who explains what they are trying to do and shows some code along with indications of what it produced (including error messages) and what it should produce. Then the question should not be a request to just FIX THIS or WRITE IT FOR ME but asking if someone can show what they did wrong with some idea where it went wrong.
>
> This may not be so common but it allows faster and easier help.

+1


...

> I will end with a comment. I have heard of interview techniques for a job where they deliberately supply a problem in which the goal is not so much to be easy to solve in front of them in real time but to watch how the person looking for a job responds to the uncertainties and asks follow-up questions or verbalizes things like, if it is like this, I might use this technique but if you also need that then ...
>
> So, I shudder to think what happens if someone being interviewed turns around and asks us and further confuses things with changes to make it harder to recognize they are asking for outside help. The answer expected may well be to NOT use say the older versions of PASCAL to do something but switch to something better suited (and for that matter available.) I would not want to program the DES encryption/decryption method in Pascal again! And these days, it seems much better to just find a module or package that meets such needs.

As you know, I investigate Cognitive Psychology. Accordingly, such is
interesting to me. In many cases, I'll interview for motivation, not
just particular skills - but perhaps that's another can-of-worms.

How about "when is 1 + 1 not 2?". This is a bit of a mind-bender, but
leads towards debugging ability* - what if your code was showing some
unbelievable result like this?

The answer is (or rather, "could be") 10, ie we're looking at binary cf
decimal, coding.

Do I hear some groans? Yes, fair-enough! There was absolutely no
"context" to the question - whereas when coding/debugging we would
expect to have some 'framing' of our thinking. At the same time, 'fixed
mode thinking' will prevent many people from even considering such
possibilities - whether as-asked or in a dev.env...


* In the ?good old days, career progression was thought to be:
(mainframe) Computer Operator, to Computer Programmer, to Systems
Analyst, etc. However, as I pointed-out (to an IBM 'big-wig' who took an
instant dislike as a result) the valued skills of an Analyst are that
(s)he can see 'patterns' - whereas the skills of debugging involve
realising why an expected pattern doesn't work (as-expected).


Another analysis might be to decide if the job requires a 'lateral
thinker' or a more single-minded approach. (us lateral thinkers tend to
ask (loads of) questions, and thus can be quite 'annoying' individuals).

Then there is the main-stay of many job-adverts: "attention to detail"
and the question of whether someone who can't [be bothered to] write an
half-decent email-message (with spell-checker likely built-in) is going
to be productive when communicating with a pedantic compiler?


Again, some people are suited to this business (or specific jobs
within), and some (?many) are not - but many are (perhaps reluctantly)
programming to get some other job done...

--
Regards,
=dn