With 0.2.37 updates to varnish vcl, can no longer stay logged in

[mcheck]

Maybe I spoke to soon about updating the provisioning. With the changes in .37 to the varnish vcls, wordpress will no longer stay logged in. I re-provisioned the vagrant with the latest 0.2.38, actually.

I can sometimes get it to the dashboard, not sure what it working (or not). But as soon as I click a sidebar element, it redirects me to login.

I've killed browser cache, varnish cache, restarted services, and even tried to revert vcl settings. No dice. But again, if I disable varnish, it works as expected.

[ericclemmons]

Alright, fixing this now. I was hoping v0.2.37 would've simplified some things, but there's a bug to be addressed here.

[mcheck]

Thanks. Do you have any idea what has changed? I reverted the wordpress.vcl file to pre-0.2.37 and I still get same errors. I was looking to see what other changes that would affect varnish, but I couldn't find anything specific.

[ericclemmons]

The sucky thing is, there are tons of reports of WordPress without Varnish having this same problem. I'm going to kill Varnish locally & see if it persists, then work backwards from there...

[mcheck]

For me, turning off varnish sudo service varnish stop
And adjust apache vhost 000-local.mysite.com and ports.conf to use port 80 and the current IP

Everything works as expected.

[ericclemmons]

Going to revert 0.2.37 for now

[ericclemmons]

v0.2.39 reverts aa9e83f

[mcheck]

Thanks for the revert @ericclemmons

I have good news. I figured out what change was causing the lost logins in the varnish configs.

This line in the production.vcl

# Remove any '_' prefixed cookies
set req.http.Cookie = regsuball(req.http.Cookie, "_[^=]+=[^;]+(; )?", "");

I re-enabled everything else, including the changes to the wordpress.vcl and it works great as long as the above is not activated.

Even better news: with the changes made to wordpress.vcl, the plugins and updates work on local! Sweet!

On my limited testing here on local, it works great. I can test on a remote staging system and see whether it is good.

Noice!

[ericclemmons]

This is all @EvanK's fault! Well, not really, but he pwns regex's on a regular basis.

We'll revert the revert & instead use Varnish's recommended version:

https://www.varnish-cache.org/docs/3.0/tutorial/cookies.html

Also, I think we owe you a gift basket for all of the help :)

[EvanK]

I think we can just prefix the regex we're using with (^|;\s*) (as it's currently matching an underscore anywhere in the cookies string).

And I concur about the gift basket 👍

[mcheck]

Hey thanks guys. Happy to contribute. :)

I'll try that regex and report results.

[mcheck]

I can confirm that
(^|;\s*)_[^=]+=[^;]+(; )?

works as the regex match string in varnish production.vcl allowing wordpress to stay logged in correctly.

So the updates to varnish in 0.2.37 can be re-instated, with the small change to production.vcl line as:

set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)_[^=]+=[^;]+(; )?", "");

@EvanK For grins, can you tell me what the (^|;\s*) does exactly? I see the group as "beginning of the string OR 0 or more strings" But I can't see what the ';' does. I plugged and played with regex interpreters for 30 min, and just gave up. Can you tell me?

[EvanK]

Since we may receive multiple cookies in potentially any order, and they are delimited by a semicolon and (typically) a single space, we are matching "beginning of string OR literal semicolon followed by zero or more whitespace characters".

That way, we properly remove underscored (private) cookies at the beginning of the batch:

Cookie:
__gaid=UA-12345678-1;
wordpress_e650c40f2ad2c36ac24e3d0319083626=user|1234567890|e650c40f2ad2c36ac24e3d0319083626;
wordpress_test_cookie=WP Cookie check;
wordpress_logged_in_e650c40f2ad2c36ac24e3d0319083626=user|1234567890|986c557f29ae89cf0a060151cb9a6c04

As well as in the middle/at the end of unrelated cookies:

Cookie:
wordpress_e650c40f2ad2c36ac24e3d0319083626=user|1234567890|e650c40f2ad2c36ac24e3d0319083626;
wordpress_test_cookie=WP Cookie check;
__gaid=UA-12345678-1;
wordpress_logged_in_e650c40f2ad2c36ac24e3d0319083626=user|1234567890|986c557f29ae89cf0a060151cb9a6c04

[mcheck]

ok, so that ; is a delimiter for the end of the (potential) previous cookie that is part of the whole stream of cookies.
That was tripping me up: I couldn't figure out why we would be checking for a semicolon at all - I had no idea those we're all delimited by semicolons. Thanks.

[mcheck]

@EvanK @ericclemmons Hey guys, I found another problem with these changes - though I'm not sure where this one is coming from. When I apply the varnish changes from 0.2.37, the admin side is REALLY slow. It takes forever to load images and stalls on many remote script grabs. I can do some further testing, but overall, I think this one has to do with wordpress.vcl and the changes to wp-admin.

The stalls are with a Unbuntu 12.04 server running on Digital Ocean. I revert the varnish changes to 0.2.39 and the stalls go away. One way to see the problem for me is have 0.2.37 varnish provisioned and try the redirect upon going to /wp-login/ Mine takes upwards of 12 seconds to load the wp image. I can send the network timing if you like.

[EvanK]

Not sure if this is related, but the cookie regex discussed above still has some issues -- notably that, since we are matching (and thus removing) semicolon+space delimiters on both sides of the cookie, it would end up only matching every other potentially offending cookie. I think the best fix would be (^|;\s*)_[^=]+=[^;]*:

(^|;\s*) # beginning of string OR literal semicolon and zero or more whitespaces
_        # literal underscore
[^=]+    # one or more characters NOT matching an equals sign
=        # literal equals sign
[^;]*    # zero or more characters NOT matching a semicolon

There are two important changes here:

1. * (zero or more) instead of + (one or more) on the character class after the literal = will catch cookies with empty values (which I saw on the site I was testing this with)
2. removed the trailing capture of semicolon + space character (which should have been \s* anyway), which will preserve trailing semicolons and allow the very next cookie to be matched properly