It’s 2020, when Google promised to shut off many third-party apps that weren’t verified by December 31, 2019. While I haven’t heard any reports of anyone being shut off yet, having been through Google’s OAuth verification process, and having run some Unverified Apps, I’ve learned the ins and outs of the whole process. All of these insights I’ve concluded on my own, thanks to Google’s insufficient documentation.

1. There’s no hierarchy to the API Scopes, and that’s just ridiculous

Let’s say you’ve been approved for the almighty https://mail.google.com Gmail API scope, which gives you full access to read, write, and delete data in a user’s Gmail account. Then later, you realize you don’t need this almighty scope — you determine you just need modify-only access to Gmail. You also want the benefit of a less scary warning for your user. You might think that you can just switch your OAuth code to use the less permissive scope, https://www.googleapis.com/auth/gmail.modify, and you’ll be good to go. That makes logical sense, after all.

But you’ll be shocked to learn that you’ve just re-entered “Unverified App” territory.

 

That’s right. Internally, Google doesn’t know that if you’ve been approved for the mail.google.com scope, which covers everything, you should also be approved for the “modify” scope, which grants a subset of permissions of the mail.google.com scope. There’s no hierarchy to the scopes as far as Google is concerned, which is absolutely ridiculous given that Google’s main business is to organize the world’s information. Funny that Google is so good at organizing trillions of web pages but is terrible at organizing a few hundred of its own API scopes.

2. Where exactly do you designate the scopes you want to use?

The whole OAuth process might be confusing to you because you’ve likely noticed that there are three places to specify the API libraries you want access:

1. In the Cloud Console, under the API Library section
2. In the Cloud Console, under your specific OAuth consent screen
3. In your code where you hit the Google OAuth2 endpoint

So which list is most important? Do you have to specify the scopes in the Cloud Console consent screen settings for your code to request access to that scope for a user? No, you don’t.

Here’s an example. In the Cloud Console for my GMass app, here’s the list of Enabled APIs:

 

My OAuth client consent screen lists these scopes:

In code, I can request access to a scope that’s not listed in either the Dashboard or my OAuth client page, and it might still work, depending on the scope that I add. For example, I can add the “https://www.googleapis.com/auth/genomics” scope, which has nothing to do with my project, and the user can still authenticate but IS presented with the Unverified App screen.

        private static readonly string[] GMass_Scopes = new[] {
        "https://mail.google.com",
        "https://www.googleapis.com/auth/userinfo.email",
        "https://spreadsheets.google.com/feeds",
        "https://www.googleapis.com/auth/genomics"
        };

        public ActionResult Login(string redir = null, bool connect = false, string emailaddress = "", string source = "extension")
        {
            OAuthState state = new OAuthState
            {
                Connect = connect,
                Redir = redir,
                Source = source
            };

            UriBuilder uri = new UriBuilder();
            uri.Scheme = "https:";
            uri.Host = "accounts.google.com";
            uri.Path = "/o/oauth2/auth";

            var qs = HttpUtility.ParseQueryString("");
            if (emailaddress != "")
            {
                qs["login_hint"] = emailaddress;
            }
            qs["access_type"] = "offline";
            qs["response_type"] = "code";
            qs["approval_prompt"] = "force";
            qs["client_id"] = OAuthHelper.ClientSecrets.ClientId;
            qs["redirect_uri"] = "https://extension.gmass.co/OAuth/AuthCallback";
            qs["scope"] = string.Join(" ", GMass_Scopes);
            qs["state"] = JsonConvert.SerializeObject(state);
            uri.Query = qs.ToString();

            return Redirect(uri.ToString());
        }

But, if I add the “https://www.googleapis.com/auth/drive.file” scope, also which is not enabled in Cloud Console’s API Library and not listed as part of my OAuth consent screen, then the user sails right through when logging in, and does not see the Unverified App screen. The “genomics” API isn’t documented as sensitive or restricted, so this difference doesn’t really make sense.

The rules are as follows:

• An API scope only needs to be listed as part of your OAuth consent screen if it requires verification to be used. And to even get it to appear on this settings page, the API Library for that scope needs to be enabled.
• If an OAuth scope does not require verification, or you choose to go Unverified, then the scope does not need to be listed on the Consent Screen settings, and your code can still request use of the scope, as I requested the use of the genomics scope in my code sample above.
• If your code doesn’t request access to a scope, and then attempts to use the API for that scope, you’ll get this error from Google:

  Google.Apis.Requests.RequestError\r\nInsufficient Permissions: Request had insufficient authentication scopes.

• If your code does request access to a scope, is granted access by the user, and then attempts to call an API for that scope, you will get an error if the API Library isn’t enabled in the Cloud Console.

The conclusion is that the scope need not be listed on your consent screen settings in order for you to request it, but the API Library does need to be enabled in order for your code to use the API once consent is granted by the user.

Bonus: What happens if your app has been verified and you need to add a scope later?

I contacted the Google OAuth team and asked them this very question. Here is the answer:

The steps to add a scope later are:

1. Add the scope to your OAuth consent screen, and hit either “Save” or “Submit for Verification” if it’s a sensitive or restricted scope.
2. The scope will now appear with the yellow warning sign.
3. Do not modify your production code to use the scope. As long as your production code only requests the scopes that have been approved, your users won’t see the Unverified app screen. So, just the act of adding a scope to your consent screen doesn’t alter what your user sees when going through the OAuth flow.
4. You’ll likely be contacted by the OAuth team requesting a YouTube video demonstrating the necessity for the new scope.
5. After you answer their questions and are approved for the new scope, then it will show up as green in the consent screen settings, and only then should you add the scope to your production code OAuth flow.

3. Sensitive Scope? Restricted Scope? Google’s documentation is god-awful.

In the OAuth Verification FAQ, Google discusses sensitive scopes vs. restricted scopes. Yet while the genius who wrote this listed the restricted scopes, he did not list the sensitive scopes. In response to the question “What are sensitive API scopes?”, the page says:

Sensitive scopes allow access to Google User Data. If an app uses sensitive scopes, it must comply with the Google API User Data Policy and have its OAuth consent screen configuration verified by Google.

The app verification process can take anywhere from 3 to 5 business days.

To add to the confusion, in the Cloud Console, you might see this for your project under the “OAuth consent screen” settings:

 

Notice Google’s mistake here? They’ve flagged the gmail.insert and gmail.readonly scopes as “sensitive scopes,” except that they’re not actually sensitive scopes. They are “restricted scopes,” according to the FAQ. If that’s not enough, in Google’s own master scope documentation, where all the Gmail API scopes are listed, there’s not a single mention of some of them being sensitive or restricted.

You might think the main scope list would be the perfect place to specify what’s sensitive, restricted, or neither, but you’d be wrong.

There is one place where you can definitely tell whether a scope is sensitive or restricted, and it’s here. All other Google pages fail to tell you, or they give you incorrect information.

4. Want to mark your app as “Internal Only?” There’s a bug in the Cloud Console.

The docs say that an owner of a Cloud Console project can mark it “Internal Only,” but I haven’t found that to be the case. In one of my currently Unverified Apps, SearchMyEmail.com, the ability to set the project INTERNAL is disabled, telling me that I can’t mark it INTERNAL because I’m not a G Suite user.

Except that I am. I’m logged in with my [email protected] G Suite account, which is an OWNER of this Cloud Console project. I suspect there’s a bug here that only allows the CREATOR of the Cloud project to mark the project “INTERNAL.” In my case, I created the project with an @gmail.com account. For now, that means I’m stuck in circular logic. If I login to the creator account, which is a gmail.com account, I can’t mark the project INTERNAL because I’m not a G Suite account. If I log in to my G Suite account to mark the project INTERNAL, I can’t because I’m only the owner, not the creator.

5. Comfortable with being Unverified? It’s not as easy as it sounds.

You may choose to forego the whole verification process for a number of reasons. Perhaps you don’t want to pay for the security assessment. Perhaps your app is just for internal use. Perhaps you are under the 100-user threshold for requiring verification. You may choose to go Unverified thinking your users just won’t care whether you’re Verified or Unverified because they trust you. That’s all well and good, but if you choose to go Unverified, there are two things to know:

First, Google intentionally makes it difficult for a user to bypass the “Unverified App” screen. Here’s the standard Unverified App screen that users see:

It informs the user that if they trust the app, they can “proceed.” But just how does one proceed? The word “proceed” isn’t linked. There’s no button that says “proceed.” Instead, the user has to click “Advanced,” and only then is the user given the option to proceed, after noting in parentheses that doing so is unsafe. Holy moly. Talk about scaring a user unnecessarily. Or should I say, scaring a developer into paying $15,000 – $75,000.

Here’s my recommended approach. On my app SearchMyEmail.com, which I’ve decided to let remain “Unverified,” we’ve built an animation showing the user exactly how to bypass the “unverified” nonsense.

Feel free to copy this technique. Just go to SearchMyEmail.com, choose to sign up as a “boss,” and then view the source for the interstitial page that has this animation.

Secondly, everything with Google OAuth is inconsistent. I should say, the only consistency with their OAuth rules is their inconsistency.

Check this out:

Sometimes, and it’s unclear when, being Unverified results in a user being unable to connect, regardless of what they do. In my SearchMyEmail.com app, two supposedly equivalent Gmail accounts, [email protected] and [email protected] behave entirely differently when authenticating into SME.

Since SME is Unverified, and since these are both @gmail.com accounts, they’re both likely to be shown extra warnings. This is what happens when [email protected] authenticates:

I’m told the app is unverified but given the option to continue.

But this is what happens when [email protected] authenticates:

I’m told the app is unverified but NOT given the option to continue.

What’s the difference between the two @gmail.com accounts? The only one I can think of is that [email protected] was created 10 years ago, and [email protected] was created 10 hours ago, but nowhere in Google’s OAuth docs is it stated that account age makes a difference.

Even weirder, I have two apps that I’ve decided to leave Unverified, Wordzen and SearchMyEmail. Here are the Cloud Console config screens from each:

SearchMyEmail’s Consent Screen

SearchMyEmail’s Scopes

Wordzen’s Consent Screen

Both show the status as “pending security assessment.” BUT, when you go into the App Details:

Well, look at that! The full https://mail.google.com scope is approved and granted. Yet the app is still labeled “pending security assessment.” I promise you Wordzen did not undergo the Security Assessment required by restricted scope apps. How did that happen? A Google miracle, perhaps. Update update update! It’s now January 25, 2020, and I spoke too soon. As I detailed on my live updates page, on January 13, I received notice from Google that Wordzen missed the deadline and use of the https://mail.google.com scope is now considered “unverified”.

Please, Google, fix this giant mess. But leave Wordzen’s awesome unfettered access alone (Never mind, they removed this access).