How to configure oAuth2 when Authorization Server is also the Resource server

I'm trying to setup a very basic oAuth2 authentication in spring boot 2.x.x using either authorization code grant or implicit grant but I can't seem to access the Resource server (which resides in the same spring boot app as the Authorization server) after the token is obtained.

Following is the configuration of WebSecurityConfigurerAdapter

@EnableWebSecurity
@Configuration
public class WebSecurityConfiguration extends WebSecurityConfigurerAdapter {

    private static final String[] IGNORE_URIS = {
            "/swagger-resources/**",
            "/swagger-ui.html",
            "/v2/api-docs",
            "/webjars/**",
            "/resources/**",
            "/h2-console/**",
            "/common/**",
            "/configuration/ui",
            "/configuration/security",
            "/error"
    };

    @Bean
    public PasswordEncoder passwordEncoder() {
        return new BCryptPasswordEncoder();
    }


    @Override
    public void configure(WebSecurity web) {
        web.ignoring().antMatchers(IGNORE_URIS);
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {

        http.authorizeRequests()
                .antMatchers("/product/**")
                .hasAnyRole("ADMIN").and()
                .httpBasic().and().formLogin().and().authorizeRequests().anyRequest().authenticated();

    }

    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.inMemoryAuthentication().withUser("admin").password("{noop}admin").roles("ADMIN");
    }

    @Bean
    public PasswordEncoder bCrypt() {
        return new BCryptPasswordEncoder();
    }

And the AuthorizationServerConfigurerAdapter

@Configuration
@EnableAuthorizationServer
public class AuthorizationServerConfiguration extends AuthorizationServerConfigurerAdapter {

    private final AuthenticationManager authenticationManager;

    @Autowired
    public AuthorizationServerConfiguration(AuthenticationConfiguration authenticationConfiguration) throws Exception {
        this.authenticationManager = authenticationConfiguration.getAuthenticationManager();
    }

    @Override
    public void configure(ClientDetailsServiceConfigurer clients) throws Exception {
        clients
                .inMemory()
                .withClient("my-client-id")
                .authorizedGrantTypes("authorization_code", "implicit")
                .authorities("ADMIN")
                .scopes("all")
                .resourceIds("product_api")
                .secret("{noop}secret").redirectUris("https://google.com").accessTokenValiditySeconds(0);
    }

    @Override
    public void configure(AuthorizationServerSecurityConfigurer oauthServer) throws Exception {
        oauthServer.tokenKeyAccess("permitAll()")
                .checkTokenAccess("permitAll()");
    }

    @Override
    public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
        endpoints.authenticationManager(authenticationManager);
    }
}

So far so good. I am able to reach the default Spring login page by typing the following Url in the browser.

http://localhost:8080/oauth/authorize?response_type=token&client_id=my-client-id&redirect_uri=https://google.com

Then The login page shows up and I enter my credentials.

After I log in I can then grant access to "my-client-id" app.

Eventually after I approve the app I can see the newly generated access token in the URL bar of the browser which is something like this.

https://www.google.com/#access_token=f2153498-6a26-42c6-93f0-80825ef03b16&token_type=bearer&scope=all

My question is that All of this flow won't work when I also configure a Resource Server.

@EnableResourceServer
@Configuration
public class ResourceServerConfiguration extends ResourceServerConfigurerAdapter {

    @Override
    public void configure(ResourceServerSecurityConfigurer resources) {
        resources.resourceId("product_api");
    }


    @Override
    public void configure(HttpSecurity http) throws Exception {
        http
                .requestMatchers()
                .antMatchers("/**")
                .and().authorizeRequests()
                .antMatchers("/**").permitAll();
    }
}

What am I doing wrong? When I try to access the oauth/authorize url as before I get the following:

Why? How can one access the login page and retrieve the token? What Am I missing?