Discussion:
gmail with alpine not anymore possible
(too old to reply)
Roderick
2020-05-04 11:12:25 UTC
Permalink
Google continously disabled acces to "less secure apps", and I
continously enabled it.

Today it was not possible to enable it, the link does not anymore
appear in the web site.

Google cost not only your privacy, but also always more time and
nerves.

Rodrigo
Roderick
2020-05-04 11:49:27 UTC
Permalink
It seems I must try Xoauth2, googles theatre.
m***@eai.eu
2020-05-04 12:41:11 UTC
Permalink
Post by Roderick
Google continously disabled acces to "less secure apps", and I
continously enabled it.
Today it was not possible to enable it, the link does not anymore
appear in the web site.
Google cost not only your privacy, but also always more time and
nerves.
Rodrigo
Hi,

this less secure apps login problem was temporary and now fixed
Roderick
2020-05-04 13:04:38 UTC
Permalink
Post by m***@eai.eu
this less secure apps login problem was temporary and now fixed
But the end of "less secure apps" with google seems to be a question
of time:

https://www.theregister.co.uk/2019/12/17/google_tightens_the_screw_on_less_secure_apps_will_block_most_access_from_june_2020/

Nice that alpine has a solution.
m***@eai.eu
2020-05-04 13:26:29 UTC
Permalink
Hello Roderick,

What is the solution for Alpine please? I know google wants to use oauth 2. Is there a solution for Alpine now? Will it be usable in the future when they will require oauth2?

Thanks for answering

Marek
Roderick
2020-05-04 13:31:54 UTC
Permalink
Post by m***@eai.eu
What is the solution for Alpine please?
I am trying this, but till now no succes:

http://alpine.x10host.com/alpine/alpine-info/misc/xoauth2.html
Adam H. Kerman
2020-05-04 18:10:29 UTC
Permalink
Post by Roderick
Post by m***@eai.eu
What is the solution for Alpine please?
http://alpine.x10host.com/alpine/alpine-info/misc/xoauth2.html
Eduardo's Web page doesn't include steps to be taken on the Google
Account side first. btw, if alpine is able to reach Gmail in background
after supplying the 16-digit passcode generated by Google, you will not
see the page with the giant URL.

Google overrides your setting "Use Less Secure Apps" for your
inconvenience, forcing you to turn it back on if you don't want to use
XOAUTH2.

Before you can use alpine as a "More Secure App", using XOAUTH2,

1) Turn on two-factor identification.

https://support.google.com/accounts/answer/185839

2) Generate a 16-digit passcode for use with alpine

https://support.google.com/accounts/answer/185833

Copy the passcode and save it in a temporary text file, but after
setting up authentication in alpine, delete that file.

3) Gmail settings for IMAP and SMTP

https://support.google.com/mail/answer/7126229

It may be possible to turn two-factor authentication off once things
are working in alpine with the generated passcode, but I haven't tried
it yet.

At this point, shut down any open sessions of alpine and restart alpine.

VERY IMPORTANT

If you are saving passwords in ~/.pine-passfile AND the passfile is
encrypted, DO NOT supply the passkey to decrypt the file. You need to
force alpine to store and overwrite your existing password with new
credentials and storage of the passcode generated above for OAUTH2.

Set up IMAP and SMTP for Gmail in SEPARATE alpine sessions.

1) Follow Eduardo's instructions for setting up a new account.

2) Set up either IMAP or SMTP in this alpine session, exit from the
session, then restart to set up the other service. Again, DO NOT supply
the passkey to decrypt your passfile.

3) At the point at which you have supplied the 16-digit passcode
generated by Google, alpine will ask if you wish to save it in your
passfile. This is the point at which you supply the passkey to decrypt
the passfile. alpine will then overwrite old settings, which you want.

4) Test

5) Quit your alpine session, restart to set up for the other service,
once again repeating DO NOT decrypt the passfile.
Eduardo Chappa
2020-05-05 05:39:08 UTC
Permalink
Post by Adam H. Kerman
Eduardo's Web page doesn't include steps to be taken on the Google
Account side first. btw, if alpine is able to reach Gmail in background
after supplying the 16-digit passcode generated by Google, you will not
see the page with the giant URL.
Adam, let me tell you the story so you can tell it later correctly. This
message became very very long. I apologize for the length, but the story
is long.

What Alpine does, when connecting to any server, is to compare the
authentication methods in Alpine with the authentication methods supported
but the server, and then sorts them in some preference system which is set
up at compile time. At the moment of this writing, the PLAIN authenticator
comes before the bearer, which comes before auth2.

This means that if Gmail allows you to use Plain, Alpine will use it.
There is no set up to make for this, except for that setting of allowing
less secure apps, but that has nothing to do with XOAUTH2. For Bearer and
XOAUTH2 you have to make the same set up.

I have never used a app password in alpine, but I imagine it is a special
password that goes to gmail using the plain authenticator (that would make
sense to me) so if you are using one already, Alpine will use that one
immediately.

In other words, if you are already using PLAIN authentication, you do not
need to shift to using OAUTH2, for example. It is only when Gmail says no
to PLAIN authentication that either Bearer or OAUTH2 come into play.

HOWEVER, users can override all of this. Version 2.22 allows you to use
the XOAUTH2 authenticator EVEN if you are already using a password
provided by Gmail, and both can coexist peacefully in your password file.
This is because internally Alpine saves XOAUTH2 passwords differently than
PLAIN authentication passwords, because they are intrinsically different
types of object, so it makes no sense to replace one by another. In
addition, there is an option to add /auth=xoauth2, or /auth=plain to the
definition of a server to force the authentication that you want. So if
you do not want to try XOAUTH2, simply add /auth=plain to your gmail
settings.

I do not know, but I suppose that an app that uses XOAUTH2 is not less
secure.
Post by Adam H. Kerman
Google overrides your setting "Use Less Secure Apps" for your
inconvenience, forcing you to turn it back on if you don't want to use
XOAUTH2.
Before you can use alpine as a "More Secure App", using XOAUTH2,
1) Turn on two-factor identification.
I wrote the code for XOAUTH2 and I never had to do this before, during or
after testing. I used a vanilla gmail account (not a company account) and
I never had to turn on two-factor identification.
Post by Adam H. Kerman
VERY IMPORTANT
If you are saving passwords in ~/.pine-passfile AND the passfile is
encrypted, DO NOT supply the passkey to decrypt the file. You need to
force alpine to store and overwrite your existing password with new
credentials and storage of the passcode generated above for OAUTH2.
As I said before, this is not necessary. I currently have passwords for
PLAIN and XOAUTH2 authentication in the password file. It is all about how
you play the game, but this is definitely not necessary. Use /auth=XXXX
instead. Much simpler.
Post by Adam H. Kerman
Set up IMAP and SMTP for Gmail in SEPARATE alpine sessions.
1) Follow Eduardo's instructions for setting up a new account.
I will have to come back to this later on. As I mentioned before, I never
had to set up anything separately. That would have been a design disaster.
No it cannot work like that. I would oppose anyone designing a system this
way. I set up the IMAP server and the SMTP server correctly, and then
login to the IMAP server first (because of tradition, not because of
need). Alpine knows internally that both servers use the same credentials,
so giving one password for one server gives the password for the other.
This is only for XOAUTH2, not for PLAIN authentication. This means that
once you set up your access using XOAUTH2 for one server, you do not have
to repeat it for the other. Alpine knows what to do.

There is more to this later, but here is the bottom line: The design of
the interface was thought such that if you simply configure the servers as
they appear in the web page, and follow the directions specified in the
web, then everything should magically work (and it does, except for
another reason which I will explain later!)
Post by Adam H. Kerman
[...]
I removed lots of text written by you about how to set up Alpine. Please
try to forget about this, and let me try to explain what works, how this
is supposed to work, and what can users do to make it work when it does
not work.

First things first: Set up your servers exactly as follows:

inbox: {imap.gmail.com/ssl/user=***@id.com/auth=xoauth2}INBOX
smtp: smtp.gmail.com/ssl/user=***@id.com/auth=xoauth2

In theory the second step is to follow the directions that appear in my
page, at

http://alpine.x10host.com/alpine/alpine-info/misc/xoauth2.html

but somehow they do not work. The reason is because Google stopped
allowing people to use the client-id and client-secret that alpine got
from Google because they put Alpine as an "unverified app". Google has
their reasons to do so. In general, it prevents that a malicious app
abuses the information it can collect from its users. Google requires that
apps be verified, and I tried to make Alpine go through the verification
process, just to find out that it is not possible for me to complete that
process, and I showed in a video to Google that what they are asking me is
not possible to do because of the way that they set up the verification
process, and even in that case I was ignored, so Google is playing big
guy, and Alpine little guy, and big guy is ignoring little guy because it
can.

Just to be clear, in case you ever hear Google's version. They were hung
up on Alpine not having a privacy policy, and not having terms of use. I
told them that their process eliminates Alpine before I could write those
terms. In other words, I showed google that even if I wrote those terms,
and I tried to go through the only path that they allowed me to go, I
would still not be able to complete the process, because the obstruction
for me was not the privacy policy, it was them rejecting the server, and
there was not way to get past that, so privacy policy or not, there was
nothing for me to do, and I quit talking to a company with deaf ears.

So, if Alpine were a verified app then the steps described in my page
about configuring Gmail with XOAUTH2 would actually work. In fact, it
works for 100 gmail accounts today, as that is the maximum number of users
Google allows for an unverified app.

While I was talking to Google about this, they suggested that Alpine users
do the same as mutt users do to use XOAUTH2 with Gmail. I will describe
this generically, and it is not for the faint of heart, but I was told
that Alpine users had the option to register Alpine as their app, and so
obtain a client-id and client-secret, which is really all that Alpine
users need in order to use Alpine with Gmail in XOAUTH2. Since every
unverified app can be used by up to 100 users, it would be possible for
every user of Alpine to register Alpine as an app with Gmail and then
share their client-id and client-secret with others and still have access
to Gmail.

So how is a person supposed to register Alpine with Gmail? Hold tight,
this is what Google wants you to do:

* Login to https://console.developers.google.com with the same
credentials that you use to login to your gmail account.
* Register Alpine. You need to allow Alpine access to the scope
https://mail.google.com/. Other scopes I have set up are "email,
profile and openid". The one that Google cares about is
https://mail.google.com. That is where the issue really is.
* Once you have registered Alpine, go to create OAUTH credentials for
Alpine. You will be given a client-id and a client-secret.
[ I did this a long time ago, and I do not recall the exact steps to
do this, but this is the correct order in which to do this.]

* Copy those values, and add them to Alpine. This is done in the screen
that you get by pressing "M S U". I had to add this screen because
Google did not allow me to submit Alpine for verification, so I had to
write the code, just before releasing Alpine, so that users could use
Alpine with XOAUTH2, albeit, not the way it was intended originally.

* Once all of this has been done, you can open an IMAP or SMTP connection
and follow the steps in my web page to configure Alpine to use XOAUTH2.

Yes, please send a thank you note to Google for making you waste your time
trying to figure out how an end user is supposed to register an app, which
is a highly technical thing. Google said that mutt users were supposed to
do that, hence Alpine users will have to do that too. It is only fair!
After all Alpine users do not have any other thing to do than trying to
figure out how to navigate the registration process, instead of writing
research papers, monitor computing systems, or just simply be
discriminated because they cannot use their own senses to access the pages
in the google site that are required to register Alpine as an app. I have
no idea how a blind person is supposed to do any of this process, and
Google could care less about leaving them out, just because they have deaf
ears, and they think that their good and bad reasons is all that matters.

So if I did not lose you. The point is very simple:

* Set up the gmail servers exactly as above
* Get some client-id and client-secret from somewhere
* Add those values to alpine in the "M S U" screen
* Go through the process in my web page to configure Alpine to use Gmail
with XOAUTH2

Alpine will take it from there. I hope some day steps 2 and 3 will not be
necessary, but today they are. Sorry for the rant, but you probably need
to know what happened so that you understand why you are wasting your time
so much trying to make this work. I tried to make it as easy as possible
for everyone, but Google trumped me.
--
Eduardo
https://tinyurl.com/yc377wlh (web)
http://repo.or.cz/alpine.git (Git)
Adam H. Kerman
2020-05-05 15:09:13 UTC
Permalink
Post by Eduardo Chappa
Post by Adam H. Kerman
Eduardo's Web page doesn't include steps to be taken on the Google
Account side first. btw, if alpine is able to reach Gmail in background
after supplying the 16-digit passcode generated by Google, you will not
see the page with the giant URL.
Adam, let me tell you the story so you can tell it later correctly. This
message became very very long. I apologize for the length, but the story
is long. . . .
Eduardo, I don't want to get into another huge back-and-forth with you.

I wrote from my own recent experience, not yours. Google, indeed, had
turned off access to "less secure apps" in my account, so I went
through XOATH2 process with some difficulty and had to turn on two-factor
authentication in order to generate the app passcode.

At no point have I expressed the opinion that there's anything wrong
with alpine. The extra difficulties and necessary trial and error are
due to requirements added by corporate owners of the mail servers, not
any problems with the client.

Also, what I ended up doing is not the only way to set up new accounts.
I did it that way after Google turned off access to "less secure apps"
in my account.

I don't understand why access to "less secure apps" wasn't turned off in
your account, but not every Google user will have the same experience,
and things may work differently in different areas.

The OP was questioning the OAUTH2 process, hence I told him what I had
gone through.

In another thread about my setup of Yahoo mail accounts, you acknowledged
the bits I'd written about not unencrypting the passfile at the start
of the alpine session AND setting up the accounts separately for IMAP
and SMPT in order to overwrite what's already stored in the passfile.

At no point have I expressed an opinion that there's anything insecure
about PLAIN authentication or that there's anything wrong with your
program. "More secure apps" and "less secure apps" are Google's words,
not mine. In truth, I did not want to use two-factor authentication at all,
which is a privacy violation as the user must give additional identifying
information to Google. I gave them a phone number they already had.

As you will recall, use of the giant URL didn't work for me when I was
trying to set up the Yahoo mail accounts, which is why I generated app
passwords within Yahoo and why I explained about generating them for use
with Gmail. You didn't criticize me for having generated the app
password in Yahoo when I wasn't able to use the giant URL.

Various mail servers have made things a lot more difficult for the user
to set up accounts in the name of "security". The way I set up the Gmail
and Yahoo accounts using app passwords was not the only method, but I
posted about my experience because alternate methods weren't working for
me.
Eduardo Chappa
2020-05-05 19:30:19 UTC
Permalink
Post by Adam H. Kerman
Post by Eduardo Chappa
Adam, let me tell you the story so you can tell it later correctly.
This message became very very long. I apologize for the length, but the
story is long. . . .
Eduardo, I don't want to get into another huge back-and-forth with you.
Actually, we both agree in most of what you said. The differences between
Yahoo! and Gmail, however, are not part of this discussion. I agreed with
you in how you solved the issues with Yahoo! but Gmail is a different
monster.
Post by Adam H. Kerman
[...]
I don't understand why access to "less secure apps" wasn't turned off in
your account, but not every Google user will have the same experience,
and things may work differently in different areas.
I don't exactly recall this. I think that it must have been turned on in
some moment, and I probably allowed less secure apps later.
Post by Adam H. Kerman
The OP was questioning the OAUTH2 process, hence I told him what I had
gone through.
I understand you, I know you mean well and shared your experience. I was
trying to tell you how this was supposed to work and how Google trumped my
intentions.
Post by Adam H. Kerman
In another thread about my setup of Yahoo mail accounts, you
acknowledged the bits I'd written about not unencrypting the passfile at
the start of the alpine session AND setting up the accounts separately
for IMAP and SMPT in order to overwrite what's already stored in the
passfile.
Yes, that is correct. That was necessary at that time. I do not believe
that a better methods existed to do that, and still, your idea is the best
in this case. I agree with your writing at the time.
Post by Adam H. Kerman
At no point have I expressed an opinion that there's anything insecure
about PLAIN authentication or that there's anything wrong with your
program. "More secure apps" and "less secure apps" are Google's words,
not mine. In truth, I did not want to use two-factor authentication at
all, which is a privacy violation as the user must give additional
identifying information to Google. I gave them a phone number they
already had.
The terminology of less secure apps is probably correct, but it can be
confused with "less than secure". Alpine will secure your login. Less
secure refers to the fact that you are using a password to login, and that
password, if guessed correctly, gives you access to many other resources.
Instead an app password is more secure because even if guessed it only
gives you access to that resource and not all resources in Google. In this
sense, Alpine is less secure, but with XOAUTH2, there is no reason to
worry about this. XOAUTH2 is even more secure than that because the access
token changes every hour, so even if you could guess it (which is almost
impossible) you would be granted limited access for one hour to the
account.
Post by Adam H. Kerman
As you will recall, use of the giant URL didn't work for me when I was
trying to set up the Yahoo mail accounts, which is why I generated app
passwords within Yahoo and why I explained about generating them for use
with Gmail. You didn't criticize me for having generated the app
password in Yahoo when I wasn't able to use the giant URL.
Hold on. There is an issue here. Why is Alpine trying to use XOAUTH2 with
a server that does not know how to login? Did you mean Gmail? Let me
investigate this. That does not make sense. Alpine should not be trying to
login using XOAUTH2 with servers that it does not know. I will look into
that.
Post by Adam H. Kerman
Various mail servers have made things a lot more difficult for the user
to set up accounts in the name of "security". The way I set up the Gmail
and Yahoo accounts using app passwords was not the only method, but I
posted about my experience because alternate methods weren't working for
me.
I understand you. The basic flow is not working in Alpine, and that is an
issue with Alpine. Unfortunately I do not have a Yahoo! account to test
anything, so all I can do is to see what you did, what you attempted to
do, and how this worked. I think your method for Yahoo is very good, but
in XOAUTH2, the flow should be, more or less, the one I ourlined
yesterday.
--
Eduardo
https://tinyurl.com/yc377wlh (web)
http://repo.or.cz/alpine.git (Git)
Lucas Levrel
2020-08-26 16:10:25 UTC
Permalink
Hi,

I'm un-burying this thread because I'm just facing the Yahoo "less secure
third-party app" warning (received yesterday in my Yahoo inbox).
Post by Adam H. Kerman
In another thread about my setup of Yahoo mail accounts, you acknowledged
the bits I'd written about not unencrypting the passfile at the start of
the alpine session AND setting up the accounts separately for IMAP and
SMPT in order to overwrite what's already stored in the passfile.
Yes, that is correct. That was necessary at that time. I do not believe that
a better methods existed to do that, and still, your idea is the best in this
case. I agree with your writing at the time.
does "at that time" mean the steps are different now? In short, how should
I connect to Yahoo in a way that pleases them, now? They offer to generate
a "third-party app password", should I simply do this and use it instead
of my regular password?

Thanks for your help.

P.-S. : the words I quote from Yahoo are translated from French so they
may not be the vanilla English terms; I think they should be
understandable however.
--
LL
Lucas Levrel
2020-08-26 16:46:00 UTC
Permalink
Post by Lucas Levrel
Hi,
I'm un-burying this thread because I'm just facing the Yahoo "less secure
third-party app" warning (received yesterday in my Yahoo inbox).
Post by Eduardo Chappa
Post by Adam H. Kerman
In another thread about my setup of Yahoo mail accounts, you acknowledged
the bits I'd written about not unencrypting the passfile at the start of
the alpine session AND setting up the accounts separately for IMAP and
SMPT in order to overwrite what's already stored in the passfile.
Yes, that is correct. That was necessary at that time. I do not believe
that a better methods existed to do that, and still, your idea is the best
in this case. I agree with your writing at the time.
does "at that time" mean the steps are different now? In short, how should I
connect to Yahoo in a way that pleases them, now? They offer to generate a
"third-party app password", should I simply do this and use it instead of my
regular password?
Thanks for your help.
P.-S. : the words I quote from Yahoo are translated from French so they may
not be the vanilla English terms; I think they should be understandable
however.
PPS: I'm currently using 2.21.999, and from "M S M" I guess my password
file is encrypted with a certificate made by Alpine. If needed, I'm OK
with simply deleting my passfile and entering all my passwords anew.
--
LL
Eduardo Chappa
2020-08-26 22:28:19 UTC
Permalink
Post by Lucas Levrel
I'm un-burying this thread because I'm just facing the Yahoo "less
secure third-party app" warning (received yesterday in my Yahoo inbox).
Dear Lucas,

I was made aware of this issue, and I am working so that everyone gets
an upgrade soon so that Alpine will work with Yahoo! servers soon.

The worse thing that could happen is that instead of using your own
password to login to Yahoo! servers, you will have to use a special
password provided by Yahoo! (this is called the app password.)

In any case, you will still be able to continue using Alpine with Yahoo!
servers.
--
Eduardo
https://tinyurl.com/yc377wlh (web)
http://repo.or.cz/alpine.git (Git)
Eduardo Chappa
2020-10-04 18:07:33 UTC
Permalink
Post by Eduardo Chappa
Post by Lucas Levrel
I'm un-burying this thread because I'm just facing the Yahoo "less
secure third-party app" warning (received yesterday in my Yahoo inbox).
Dear Lucas,
I was made aware of this issue, and I am working so that everyone gets
an upgrade soon so that Alpine will work with Yahoo! servers soon.
Update: I contacted Verizon, who owns Yahoo!, and asked them to authorize
me to implement XOAUTH2 authorization to their Yahoo! server. They granted
me such authorization and now everyone can use XOAUTH2 authentication with
Yahoo!

The implementation is in the git repository, and I will keep testing it,
but feel very confident about it, since it did not involve much extra code
to implement. I will release it officially soon, ina new version of
Alpine.

If you test it, please send me all feedback you have about it.
--
Eduardo
https://tinyurl.com/yc377wlh (web)
http://repo.or.cz/alpine.git (Git)
Roderick
2020-05-06 06:03:30 UTC
Permalink
I thank Adam and Eduardo for these hints, and specialy very much
Eduardo also for mantaining Alpine up to date with all these
strange technic.

At the meantime the "less secure apps" reapeared in Google. But we are
warned that they may definitively disappear and we must be prepared,
perhaps to change the email provider. It is really ununderstandable
why CMU and UW abandoned the development of their imap servers to
put themselves at the mercy of these companies.

When I have time, I will follow the instructions of Eduado, because
that is what google wants and the probability of new troubles should
be less.

Thanks
Rodrigo
Loading...