Feature Request - Need Something like HttpApplication (that's pooled and assocaited to a thread)

[matlus]

In old ASP.NET we had HttpApplication (Global) that was tagged to a thread which in turn was pooled by ASP.NET. So effectively, there was one instance of HttpApplication for each pooled thread and thus:

1. HttpApplication was pool
2. We had an instance of HttpApplication that was automatically associated to a thread and so no overhead was incurred synchronizing an instance of HttpApplication to a request thread (in our application) or accessing this instance from within an ASP.NET application such as a WebForm, Controller or Handler.

This was very useful, because if we need to pool any of our own classes we could simply create an instance of this class in Global (typically in the Init() method) and assign it to a custom property of Global and this property was directly accessible from a WebForm, Handler or (WebAPI) Controller like so

((Global)HttpContext.Current.ApplicationIstance).MyProperty

Granted all pooled "things" need to be stateless or need to zero out their state at the beginning of every request. The benefit of being able to something like this is:

1. Throughput and Scale
2. Reduced pressure on the GC
3. Improved performance

If provided out of the box - makes it really easy for folks to get these benefits (caveat emptor), just like we were able to do riding on HttpApplication in old ASP.NET

I hope all of this makes sense?

Some code examples below, just in case I've not been clear enough on how I used this in old ASP.NET

The code below shows how one would initialize these pooled objects in Global. The QInsightManager is the property the makes the "pooled" object available to WebPages, Handlers, Controllers etc.

public class MvcApplication : System.Web.HttpApplication
{
    private QInsightManager _qInsightManager;

    internal QInsightManager QInsightManager
    {
        get { return _qInsightManager; }
    }

    public override void Init()
    {
        base.Init();
        _qInsightManager = new QInsightManager();
    }
}

The code below show how this pooled object can be consumed on an MVC ControllerBase class and made available to all descendants very simply.

internal QInsightManager QInsightManager
{
    get { return ((MvcApplication)HttpContext.ApplicationInstance).QInsightManager; }
}

This implementation ensures that we grab the instance associated with the instance of Global that is handling the request.

[davidfowl]

1. We had an instance of HttpApplication that was automatically associated to a thread and so no overhead was incurred synchronizing an instance of HttpApplication to a request thread (in our application) or accessing this instance from within an ASP.NET application such as a WebForm, Controller or Handler.

I'm not sure it was associated with a thread (that wouldn't work for async requests as they hop threads all the time) but @GrabYourPitchforks might know better. (ref https://github.com/Microsoft/referencesource/blob/master/System.Web/HttpApplicationFactory.cs)

This was very useful, because if we need to pool any of our own classes we could simply create an instance of this class in Global (typically in the Init() method) and assign it to a custom property of Global and this property was directly accessible from a WebForm, Handler or (WebAPI) Controller like so

What kind of objects did you need to associate with the HttpApplication? Those that didn't have any request affinity? If not, do, why not just make them singletons. If they do have request affinity, how do you zero them out?

We're not going to build this into the framework but you can add a middleware that basically does the same thing. Pooling objects requires care so things don't leak across requests (as you may already know) and we're ok making it possible for you to pool objects but there no reason to build something like this in when it can be implemented in another package.

[matlus]

HttpApplication was pooled. May not have been pinned to a particular thread for its lifetime. But ASP.NET did pool them and associated one instance with the thread on which the incoming Request was made. I've used this "feature" since ASP.NET 1.1. IHttpHandler has a property IsResuable, which when set to true would also be pool the handler instance (which is also something I use all the time). We need to do all we can to reduce pressure on the GC.

Yes, as stated in my post, one had to be aware that these objects are pooled and thus need to be stateless or one has to ensure all state is reset at the start of every request. But that's no reason to not used such a feature given the huge performance and throughput gains to be had.

And you're correct. When doing async work, one has to be aware that when as task returns asynchronously it is most likely on a different thread. Many ways to handle this scenario. Again, no reason to support this feature.

What kind of objects did you need to associate with the HttpApplication? Those that didn't have any request affinity? If not, do, why not just make them singletons. If they do have request affinity, how do you zero them out?

My entire application (barring Controllers and the like) , that is all object instances are pooled "effectively". Except for models, DTOs and the like (input and output parameters). Basically, a Controller/Handler/Page accesses the Façade via the code show earlier. That's it, from then on I am in a stateless world. All classes are stateless all the way to the database and back.

Stateless is not "exactly" equal to a Singleton. A Singleton, also implies the need to be threadsafe. Which pooled objects generally are if they are stateless, but there are caveats...

This is a breaking change for me if there is no out of the box way of doing this. I'm simply asking for a feature that has been in ASP.NET since 1.1 and something that was used by ASP.NET and exposed to us (for example the IHttpHandler).

In fact, I was going to also ask if we could have pooled Controlers (using the IsReusable property like in IHttpHandler). My controller are also stateless (in the way they are written) but I'm still paying the penalty of adding pressure to the GC .

Yes, I know I can do anything I need with any given codebase. I'm asking for a "hook" (if you will) that I can use (similar to HttpApplication), that I can use.

we're ok making it possible for you to pool objects

Yup, that's what I'm asking for 😏

[davidfowl]

HttpApplication was pooled. May not have been pinned to a particular thread for its lifetime. But ASP.NET did pool them and associated one instance with the thread on which the incoming Request was made. I've used this "feature" since ASP.NET 1.1. IHttpHandler has a property IsResuable, which when set to true would also be pooled (which is also something I use all the time). We just need to do all we can to reduce the pressure on the GC.

To understand what feature you need and what can be done I need to understand if you are just looking to pool objects that have request affinity. If the answer is yes then we can move on, but the previous post made it seem like you were talking about thread affinity.

Yes, as stated in my post, one had to be aware that these objects are pooled and thus need to be stateless or one has to ensure all state is reset at the start of every request. But that's no reason to not used such a feature given the huge performance and throughput gains to be had.

You can do this on your own, like I said, we're not adding this to the framework via some object like http application. All of the building blocks are there though.

This is a breaking change for me if there is no out of the box way of doing this. I'm simply asking for a feature that has been in ASP.NET since 1.1 and something that was used by ASP.NET and exposed to us (for example the IHttpHandler).

ASP.NET 5 is a breaking change from all of ASP.NET. There is no IHttpHandler, HttpApplication etc nor will they be. I'm just trying to get to the core of what you're asking for instead of asking for what was in the original ASP.NET because it won't look the same.

Yes, I know I can do anything I need with any given codebase. I'm asking for a "hook" (if you will) that I can use (similar to HttpApplication), that I can use.

I'm still not 100% on what you're asking for (maybe you can clarify in the chat room https://jabbr.net/#/rooms/AspNetvNext or https://gitter.im/aspnet/Home). The "hook" you have is dependency injection and maybe a piece of middleware (but I'm not even sure you need middleware).

[davidfowl]

Here's an example using the DI system:

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        // .. More things here

        // Incurs an object allocation per scope anyways
        services.AddScoped(typeof(PooledObject<>));
    }

    // .. More things here
}



public class MyController
{
    private readonly ThingIWantToPool _thing;

    public MyController(PooledObject<ThingIWantToPool> pooled)
    {
        _thing = pooled.Value;
    }
}

public class ThingIWantToPool
{

}

public class PooledObject<T> : IDisposable where T : class, new()
{
    private static readonly ObjectPool<T> _pool = new DefaultObjectPoolProvider().Create<T>();

    public PooledObject()
    {
        Value = _pool.Get();
    }

    public T Value { get; }

    void IDisposable.Dispose()
    {
        _pool.Return(Value);
    }
}

In the above example, the PooledObject<T> is allocated per scope (there's a default request scope) and is used to control the lifetime of the T that it wraps. I'm using our ObjectPool from https://github.com/aspnet/Common/tree/dev/src/Microsoft.Extensions.ObjectPool. It's referenced by project.json:

{
  "version": "1.0.0-*",
  "compilationOptions": {
    "emitEntryPoint": true
  },

  "dependencies": {
    "Microsoft.AspNet.IISPlatformHandler": "1.0.0-rc1-final",
    "Microsoft.AspNet.Server.Kestrel": "1.0.0-rc1-final",
    "Microsoft.AspNet.Diagnostics": "1.0.0-rc1-final",
    "Microsoft.Extensions.ObjectPool": "1.0.0-*"
  },
  ...
}

[matlus]

Thread affinity or Request affinity depends on the implementation because from the point of view of the Application, they both look the same.

I’m going into details to jot down some things only so we’re on the same page (conceptually) and don’t mean to suggest you don’t know this stuff 😃

I’m assuming that in the framework you manage a pool of threads (managed threads) that get associated to the TCP connection (or whatever IPC method you use) from the web server. This association will need to be done using some form of thread synchronization (lock for example).
Now if you had a ThreadContext object (in addition to HttpContext) that is associated to each of your threads at the time the thread was created (rather than per request) then one could potentially attach other “things” to this context once at the time the thread is created by the pool. The ThreadContext would be disposed when the thread pool determines it no longer needs the thread. If once could have a means to access this ThreadContext (only) in order to associate other things we wanted pooled, then I’d called this Thread Affinity

Another way would be that the framework provides hooks such that we can provide it with an xxxFactory and it can create instances and associate them with whatever is pooled and associated to each request (and dispose of them as it sees fit). This would require thread synchronization to be in play as well. I’d call this Request Affinity.

Ok, I’m hoping conceptually we’re on the same page at this point
Request Affinity is probably easier to support but support for Thread Affinity would be fantastic! I’m not sure what we have currently in the framework that stays alive (in a pool) and gets associate to each request.

I’ve implemented (outside of the framework) the Thread Affinity model based solution. Works as expected but if such a feature is provided, a lot of folks could use this without having to roll their own.

[matlus]

Just saw your earlier post after I replied...
Thanks for the code/sample. Yes, I know I can do it this way. Some (additional) thread synchronization is needed in the DefaultObjectPoolProvider. As in the previous post, I'm guessing some form of thread synchronization is already being done once during the lifetime of the Request within the framework. I'd like to be able to plug into that. That way, any number of types can be associated to the Request with only one lock (at least for this purpose).

With the thread affinity model, once something is associated to the thread I never have to synchronize for that thread again since that object is always available. I also don't have my own pool since the threads are pooled I'm able to use that same pool for my instances.

I guess because I'm using the word "pool" you're thinking I want to do the pooling 😄. While I want to piggy back on the pooling you already have.

[davidfowl]

Now if you had a ThreadContext object (in addition to HttpContext) that is associated to each of your threads at the time the thread was created (rather than per request) then one could potentially attach other “things” to this context once at the time the thread is created by the pool. The ThreadContext would be disposed when the thread pool determines it no longer needs the thread. If once could have a means to access this ThreadContext (only) in order to associate other things we wanted pooled, then I’d called this Thread Affinity

I still don't know why you care about threads in particular. Sounds like a BCL feature as ASP.NET isn't managing threads. We're just using the .NET Thread pool.

Another way would be that the framework provides hooks such that we can provide it with an xxxFactory and it can create instances and associate them with whatever is pooled and associated to each request (and dispose of them as it sees fit). This would require thread synchronization to be in play as well. I’d call this Request Affinity.

You can already do that today in about 5 different ways. The "hooks" you need are available.

Thanks for the code/sample. Yes, I know I can do it this way. Some (additional) thread synchronization is needed in the DefaultObjectPoolProvider.

Don't think so. Actually I think this does exactly what you need.

With the thread affinity model, once something is associated to the thread I never have to synchronize for that thread again since that object is always available. I also don't have my own pool since the threads are pooled I'm able to use that same pool for my instances.

That's not how any of this works. That's not how ASP.NET 4.x works either. You seem to be mixing up the general concept of a thread pool or maybe .NET's thread pool with the lifetime of pooled objects. HttpApplication instances were "pooled" and had no thread affinity; they were reset per request.

With the thread affinity model, once something is associated to the thread I never have to synchronize for that thread again since that object is always available. I also don't have my own pool since the threads are pooled I'm able to use that same pool for my instances.

See my earlier response about ASP.NET and thread management.

I guess because I'm using the word "pool" you're thinking I want to do the pooling 😄. While I want to piggy back on the pooling you already have.

Still doesn't' make sense to me. We don't pool anything so you can't piggy back on anything.

The code I sent is pretty close to what you want. Like I said before, we're not going to built in any concept of pooling but the hook already exists (and now you have a sample).

[matlus]

ok, I've tried explaining what I need in many different ways and I can't seem to get through. You've got fixated on something that I know I don't need.

The code I sent is pretty close to what you want. Like I said before, we're not going to built in any concept of pooling but the hook already exists (and now you have a sample).

Yea, I know how that works but honestly that's not what I'm asking for. Thanks for your patience David. I mean dude, seriously. You've been up at really insane hours and you've been replying promptly. Very much appreciated. 🙇 .

I'll just use what I have and I'm not managing a pool and yet my objects pooled in the context of threads so once associated I never need to lock and associate

[davidfowl]

ok, I've tried explaining what I need in many different ways and I can't seem to get through. You've got fixated on something that I know I don't need.

That's unfortunate.

I'll just use what I have and I'm not managing a pool and yet my objects pooled in the context of threads so once associated I never need to lock and associate

How you choose to pool things is up to you. You don't have to use Microsoft.Extensions.ObjectPool, you can use whatever lifetime you need. The appropriate begin/end request hooks are there for you to reset objects as needed.

For my own knowledge, a code sample of how you are "pooling" would be helpful

[matlus]

That's unfortunate.

No worries. I'll be back 😸

Yes, I'm using ThreadLocal. In this way:

1. I don't have to manage a pool
2. Once an instance is associated to a thread it is always available (automatically) without having to re-associate an instance from a pool (and have to use some form of synchronization) for every request to the application. This is done/managed by ThreadLocal and I find it to be extremely performant.

Here is what my class looks like. So far I've found this to be dependable and extremely performant. I plan on comparing this with what we get in ASP.NET old (since I have a number of large production systems with load and scalability tests already in place. I can remove the code I "typically" have in Global replace it with this class. I may need to make this class static so I don't have to create instance of this class for every request (since Controllers are not re-used across requests). One more instance of a type should still keep my comparison apples to apples.

    public abstract class ThreadLocalContextProvider<T> : IDisposable
    {
        private ThreadLocal<T> _threadLocal;

        private bool _disposed = false;

        public ThreadLocalContextProvider(Func<T> factoryMethod)
        {
            _threadLocal = new ThreadLocal<T>(() =>
            {
                T instance = factoryMethod();                
                return instance;
            });
        }

        public T GetThreadLocalInstance()
        {
            return _threadLocal.Value;
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        private void Dispose(bool disposing)
        {
            if (!_disposed && disposing)
            {
                if (_threadLocal != null)
                {
                    _threadLocal.Dispose();
                    _threadLocal = null;                    
                }

                _disposed = true;
            }
        }
    }

[simonmurdock]

Won't all this thread-specific code barf when you start using anything async?

[matlus]

No, why would it?

It's true that when an async task returns it's most likely on a different thread. But it's a thread. And this class will give you an instance (either previously created for that thread or a new one depending on whether the thread is brand new or one that has been used before). Either ways, you'll always have an instance of T issued to you by this class. It's important that your T is stateless.

The key is to always ask this class for an instance when the async task returns rather than storing a local variable for the T on your end .

The way I use this is something like this.

I have a base controller that looks like this:

    public abstract class ControllerBase: Controller
    {
        private readonly DomainFacadeProvider _domainFacadeProvider;

        protected ControllerBase(DomainFacadeProvider domainFacadeProvider)
        {
            _domainFacadeProvider = domainFacadeProvider;
        }

        protected DomainFacade DomainFacade
        {
            get
            {
                return _domainFacadeProvider.GetThreadLocalInstance();
            }
        }
    }

Notice the DomainFacadeProperty. Also in this case the _domainFacadeProvider is just a closed generic type (descendant of ThreadLocalContextProvider<T>, the class in the previous post).

Now in say my HomeController (which is a descendant of ControllerBase)

        public async Task<IActionResult> Index()
        {
            var id = await GetData();
            return Content("Here's your data: " + DomainFacade.GetSomeData());
        }

I'm using the property, but the property accessor is getting it from the ThreadLocalContextProvider<T>. Basically anywhere you need an instance of T, just use the property and you'll be fine. Just don't store a reference to T (in order to try and optimize).

[benaadams]

@matlus

If you use it local to the method and return it at the end you can even go across await points with something like

public class PooledObjectFactory<T> where T : class, new()
{
    [ThreadStatic]
    private static T _pooledObject;

    public T Get()
    {
        if (_pooledObject != null)
        {
            var pooledObject = _pooledObject;
            _pooledObject = null;
            return pooledObject;
        }
        return new T();
    }

    public void Return(T pooledObject)
    {
        if (_pooledObject == null)
        {
            _pooledObject = pooledObject;
        }
    }
}

public void ConfigureServices(IServiceCollection services)
{
    services.AddSingleton(typeof(PooledObjectFactory<DomainProvider>), 
        typeof(PooledObjectFactory<DomainProvider>));
}

public class MyController
{
    private readonly PooledObjectFactory<DomainProvider> _domainProviderPool;

    public MyController(PooledObjectFactory<DomainProvider> pool)
    {
        _domainProviderPool = pool;
    }

    public async Task<IActionResult> Index()
    {
        var id = await GetData();
        var provider = _domainProviderPool.Get();
        var content = Content("Here's your data: " + provider.GetSomeData());
        _domainProviderPool.Return(provider);
        return content;
    }
}