[HCF] newcount

[nramirezuy]

Right now Frontier can handle multiples frontier/slot but newcount doesn't reflect this. I wonder if this can be achieved in a better way.

What I did in a project:

class Frontier(_Frontier):

    def __init__(self, *args, **kwargs):
        super(Frontier, self).__init__(*args, **kwargs)
        self.newcount = defaultdict(int)
        self._cache = {}

    def _writer_callback(self, response):
        # http://localhost:8002/hcf/78/test/s/example.com
        if response.url not in self._cache:
            splited_url = url_query_cleaner(response.url).split('/')
            frontier, slot = splited_url[5], splited_url[7]
            self._cache[response.url] = (frontier, slot)
        self.newcount[self._cache[response.url]] += json.loads(response.content)["newcount"]