abstract: ScotGrid is a distributed Tier-2 centre in the UK with sites in Durham, Edinburgh and Glasgow. ScotGrid has undergone a huge expansion in hardware in anticipation of the LHC and now provides more than 4MSI2K and 500TB to the LHC VOs. Scaling up to this level of provision has brought many challenges to the Tier-2 and we show in this paper how we have adopted new methods of organising the centres, from fabric management and monitoring to remote management of sites to management and operational procedures, to meet these challenges. We describe how we have coped with different operational models at the sites, where Glagsow and Durham sites are managed "in house" but resources at Edinburgh are managed as a central university resource. This required the adoption of a different fabric management model at Edinburgh and a special engagement with the cluster managers. Challenges arose from the different job models of local and grid submission that required special attention to resolve. We show how ScotGrid has successfully provided an infrastructure for ATLAS and LHCb Monte Carlo production. Special attention has been paid to ensuring that user analysis functions efficiently, which has required optimisation of local storage and networking to cope with the demands of user analysis. Finally, although these Tier-2 resources are pledged to the whole VO, we have established close links with our local physics user communities as being the best way to ensure that the Tier-2 functions effectively as a part of the LHC grid computing framework..
  .
  .
  *17$^{\underline{\rm{th}}}$ International Conference on Computing in High Energy and Nuclear Physics*
  *Prague, Czech Republic*
author:   Sam Skipsey $^1$, David Ambrose-Griffith $^2$, Greig Cowan $^3$ , Mike Kenyon $^1$, Orlando Richards $^3$,
Phil Roffe $^2$, Graeme Stewart $^1$
$^1$ University of Glasgow, University Avenue, Glasgow G12 8QQ
$^2$ Department of Physics, Durham University, South Road, Durham DH1 3LE
$^3$ Department of Physics, University of Edinburgh, Edinburgh, EH9 3JZ
title: ScotGrid: Providing an Effective Distributed Tier-2 in the LHC Era

# Introduction

Unlike many other regions, the UK treats "Tier-2" centres as primarily administrative groupings; UK-style "distributed" Tier-2s consist of multiple sites, grouped by region (e.g. *ScotGrid* for mostly Scottish sites, *NorthGrid* for those in the north of England, &c), but not abstracted at this level to present a single "Tier-2 site" to the grid. Each component site is visible as a distinct cluster and storage, with its own Compute Element, Storage Element, Site-BDII and other components. Each component site thus also has its own systems administration teams at the host location (mostly Universities), and has to abide by the host's requirements. Hierarchical control then passes through a "Tier-2 Coordinator" who is responsible for sheparding the Tier-2 as a whole. The ScotGrid Tier-2 thus consists of sites hosted by the Universities of Glasgow, Edinburgh and Durham, administered in the manner described above. Due to the lack of an established term for component sites of a distributed Tier-2, we will adopt the common terminology in the UK itself and refer to components as "Tier-2 sites" themselves, despite the ambiguity this introduces.

This paper will attempt to provide an overview of challenges and solutions we have encountered whilst establishing the Tier-2 and scaling it to provide the expected demand from the LHC.

# Fabric and People Management

ScotGrid has adopted several common best practices for the administration of component sites. For configuration and fabric management, the Glasgow and Durham sites have adopted cfengine as an automation tool. Durham's configuration is derived from the original configuration scripts used by Glasgow, but omits low-level (non-cfengine) tools for imaging nodes and services.

All sites use the Ganglia fabric monitoring tool to collect historical and real-time data from all nodes in their sites. We have also configured a centralised nagios alerting system, monitoring Glasgow's systems directly, and Edinburgh and Durham via external tests (including SAM test monitoring provided by Chris Brew of RAL).

When failures or system problems are detected, a shared login server, hosted at Glasgow, allows sysadmins from any of the sites to log in with administrative rights to the other sites. This system is managed by ssh with public key security and host-based authentication, and is therefore fairly secure against external attacks.

Recently, we have also established a "Virtual control room", using the free, but proprietary Skype VoIP and text messaging system. The control room, implemented as a "group chatroom". has persistent historical state for all users who are members, allowing the full history of conversations within the room to be retrieved by any user on login (including conversations that occurred during periods they were not present). We believe that this feature is not available as standard functionality in any other application. The virtual control room has already proved of great value, enabling advice and technical assistance to be provided during periods of critical service downtime. As the Tier-2 is distributed, so is the expertise; a common control room allows this expertise to easily be shared in a way that community blogs and mailing lists can't replicate.

## Local services

The gLite middleware has a natural hierarchical structure, such that information flows up from all services into a small number of "top-level" services ("Top-level BDIIs") with global information, and similarly control and job distribution flows out from a small number of "Workload Management Services" to all the compute resources. As such, it was originally envisaged that the Top-level BDII and WMS services would exist at the Tier-1 level. However, for historical reasons, and due to the unreliability of WMS services, ScotGrid manages its own copies of both services, which are used by all the sites in the Tier-2. Top-level BDIIs are easy to administer, and do not require any complicated configuration; they are essentially ldap servers with some tweaks applied.

The glite WMS (technically two services: the WMS itself and the "Logging and Bookkeeping" service that maintains records) has developed a reputation for being hard to administer and maintain. The ScotGrid WMS at Glasgow was set up in order to reduce our dependance on the (then single) WMS instance at the Tier-1, removing the single point of failure this represented for job submission. Since then, the RAL Tier-1's WMS service has become somewhat more reliable, with the aid of extensive load-balancing; however, the WMS as a service is still the least stable part of the job control process, and is still a major cause of test failures against sites. Other Tier-2s have begun installing their own "local" WMSes in order to reclaim control of these test failures, as it is psychologically stressful to experience failures outside of your influence.

In addition, Durham run their own WMS instance for use by local users and for training purposes. This is not enabled as a service for users outside of Durham currently; it is mentioned here to avoid confusion in the later discussion of Durham system implementation.

Glasgow also runs its own VOMS server, although not to duplicate central services at the Tier-1 or RAL. ScotGrid supports several local VOs, one of which will be discussed later in this paper, and the VOMS server is necessary to support user membership, authentication and authorisation control for those entities. Possessing a VOMS server gives us additional flexibility in supporting local users, especially those who are interested in merely "trying out" the grid, without wanting to invest in the overhead of creating their own VO.

# "Special cases"

Whilst Glasgow's configuration is generally "standard" in terms of common practice at other UK Tier-2s, Durham and Edinburgh both have nonstandard aspects to their site configuration. Durham host their front-end services on virtual machines; Edinburgh's compute provision is via a share of a central university service, rather than a dedicated service for WLCG.

## Durham: Virtual Machines

In December 2008, Durham upgraded their entire site with the aid of funding from . As part of this process, the front-end grid services were implemented as VMware virtual machines on a small number of very powerful physical hosts.

Two identical hosts, "grid-vhost1.dur.scotgrid.ac.uk" and "grid-vhost2.dur.scotgrid.ac.uk", configured with dual quad-core Intel Xeon E5420 processors, 16 Gb RAM and dual (channel-bonded) Gigabit ethernet. The services can therefore be virtualised easily in "blocks" of (1 core + 2 Gb RAM), allowing different services to be provisioned with proportionate shares of the total host. Using VMware Server 2.0 limits the size of a given virtual machine to two cores, which does present a practical limit on the power assignable to any given service. In many cases, this can be ameliorated by simply instantiating more than one VM supporting that service, and balancing load across them (this works for compute elements each exposing the same cluster, but storage elements cannot share storage and so cannot load balance in the same way).

The current configuration allocates one block each of grid-vhost1 to VMs for ganglia monitoring, cfengine and other install services, and the glite-mon service, two blocks to the VM hosting the first Compute Element. Two blocks plus the remaining 4Gb of RAM to the local Durham WMS service, as WMS services are extremely memory hungry. This leaves one core unallocatable to additional VMs as there is no unallocated memory to assign.

Grid-vhost2 devotes one block each to the Site-level BDII service and the local resource management system (Torque/Maui), two blocks to a second CE service to spread load, and two cores and 2Gb ram to a storage element service using DPM. As will be shown later in the paper, this configuration does not provide sufficient compute power to the storage element, but load-balancing is non-trivial for DPM and other commonly implementations of the SRM protocol, so performance increases will have to be attempted via configuration tuning initially.

In all cases, the virtual machine files themselves are hosted on NFS filesystems exported from a master node, via dual (channel-bonded) Gigabit ethernet. The local filesystem is mounted on 2TB of fibre-channel RAID for performance and security. Interestingly, exporting this filesystem over NFS gave better performance than the local disks in the virtual machine hosts. As the virtual machine files are hosted on a central service, it is trivial to move services between the two virtual machine hosts, in cases where one host needs maintenance.

Outside of ScotGrid, the Oxford site (part of SouthGrid) has also installed front-end services as virtual machines. Considering the growing acceptance of virtualisation as a means of supporting multiple applications efficiently on the increasingly multicore architectures of modern CPUs, we see this becoming a mainstream configuration across the WLCG in the next five years.

## Edinburgh: Central Services

Historically, Edinburgh's contribution to ScotGrid has been storage-heavy and compute-light. This was a legacy of the initial experiment of the ScotGrid configuration, where Glasgow was intended to host compute services and Edinburgh storage, with fast networking allowing these to act as mutually local resources. This turned out to not be feasible at the time, although NorduGrid has effectively proved that such a scheme can succeed, albeit with their own middleware stack rather than gLite.

As a result, the paucity of Edinburgh's installed cluster resources lead to a decision, in 2006, to decommission the old site, replacing it with a new site with compute power provided by the then-planned central university compute resource, ECDF (the Edinburgh Compute and Data Facility). As the ECDF is a shared university resource, its main loyalty must be to the local University users, rather than to the WLCG. This is a currently unusual, but increasingly common state of affairs, and requires some adjustment of expectations on both sides.

In particular, as local users can't be forced to use the grid middleware to submit their jobs (preferring direct acces), the normal installation process for worker nodes can't be performed as they pollute the local environment with non-standard libraries, services and cron jobs. Instead, the "tarball" distribution of the glite-WN is deployed, via the cluster's shared filesystem. However, even the tarball distribution requires cron jobs to be installed to maintain the freshness of CRL files for all supported Certificate Authorities; these are instead run from the Compute Element, which has access to the instance of the CA certificate directory on the shared filesystem.

Services like the Compute and Storage Elements all run on services which are physically administered by the ECDF sysadmins, but software administered by a separate "middleware team'' who were partly funded by GridPP to provide this service. As a result, the installation and configuration of these "front-end nodes" is broadly similar to that in other Tier-2 sites; the sole exception being the interface to the local resource management system, in this case Sun Grid Engine. Again, the default configuration procedure for gLite middleware via YAIM involves configuration of the LRMS as well as the CEs that attach to it; this is highly undesirable in the case of a shared resource, as the queue configuration has usually already been arranged correctly. As a result, configuration of the CEs was performed partly by hand, and required some recoding of the jobmanager scripts provided by the gLite RPMs.

Whilst the resulting system is functional, the necessity of not using YAIM to configure some elements of the compute provision (both CE and worker nodes) reduces the agility of the site in response to middleware updates. Despite this, Edinburgh/ECDF has managed to be a successful Tier-2 site, after all the pitfalls were worked through.

# WLCG/EGEE Use

The configuration and management procedures mentioned in the previous section have enabled ScotGrid to provide significant, and reliable, resources to the WLCG VOs. The workload presented by these VOs can be divided into two main classifications: *Monte-Carlo production*, the generation of simulated datasets for use in later analysis of detector data; and *user analysis*, which comprises all activities involving "real'' data from the LHC itself. Production work by the LHCb and ATLAS VOs has been on-going on ScotGrid sites for several years, and is well-understood; User Analysis, conversely, has been necessarily limited by the delayed operation of the LHC itself. The LHCb compute model reserves Tier-1 sites for user analysis processes, and so we would only expect significant analysis activity from the ATLAS VO when the LHC begins operation.

## Production

As Production work is well established, all VOs have detailed historical logs which can be accessed to investigate the relative effectivness of sites. We will briefly discuss some statistics from the LHCb and ATLAS VOs regarding recent Production work as concerns ScotGrid sites.

### LHCb

Figure shows the number of successful LHCb production jobs accumulated by ScotGrid sites over a 6 month period ending in the 11th week of 2009. The point at which Durham's site became active is clearly visible as the rapidly increasing green area to the right-hand side of the plot.

Between September 2008 and February 2009 inclusive, ScotGrid had processed almost 90,000 LHCb jobs; 20,000 of those arrived at Durham in February alone, although the "bursty" nature of LHCb production demand prevents us from concluding that Durham is as dramatically better than the other sites as this would initially seem. Indeed, almost half of the entire cumulative distribution occurs in the month of February, a reflection more on the VO's increased demand than on Tier-2 itself.

### ATLAS

Table shows the statistics for ATLAS Production jobs passing through ScotGrid sites during the middle of March 2009. As can be seen, Glasgow processed almost 18% of the total ATLAS production workload in the sampled period. Considering that this data includes the Tier-1, which handled almost 50% of the total, this is a particularly impressive statistic for ScotGrid. By comparison, Edinburgh is comparable to the average Tier-2 site, and Durham happens to look particularly unimpressive in this accounting because of on-going problems with ATLAS jobs not targeting their site. Even with these problems, Durham, in common with the other component sites, surpasses the average efficiencies for sites in the UK significantly.

A more recent sample of the state of ATLAS production, presented in table shows that after Durham's initial teething problems were fixed, it is fully capable of matching the performance of other UKI sites. Indeed, in this sampling, all of the ScotGrid sites surpass the average job (walltime) efficiency for the UK.

| Site | Success | Failure | Walltime (success) | Efficiency | Walltime Efficiency |
|:--:|---:|---:|---:|:--:|:--:|
| Glasgow | 87376 | 16461 | 1859086649 | 84.1% | 96.4% |
| Edinburgh | 9103 | 1421 | 135756242 | 86.5% | 94.8% |
| Durham | 1427 | 113 | 504188 | 92.7% | 96.0% |
| Total UK | 489889 | 111219 | $\sim$`<!-- -->`{=html}9$\times$`<!-- -->`{=html}10$^9$ | 81.5% | 88.9% |

ATLAS Production (March 09). Walltime figures are given in minutes,"Success" and "Failure" values are total numbers of jobs.

| Site | Success | Failure | Walltime (success) | Efficiency | Walltime Efficiency |
|:--:|---:|---:|---:|:--:|:--:|
| Glasgow | 17970 | 629 | 379671909 | 96.6% | 99.6% |
| Edinburgh | 4019 | 581 | 48351886 | 87.4% | 97.7% |
| Durham | 6069 | 454 | 54405200 | 93.0% | 94.9% |
| Total UK | 294251 | 35696 | $\sim$`<!-- -->`{=html}4$\times$`<!-- -->`{=html}10$^9$ | 89.2% | 92.9% |

ATLAS Production (April 09). Taken from the first 21 days of April, reported in ATLAS Prodsys tool. Walltime figures are given in minutes,"Success" and "Failure" values are total numbers of jobs.

## ATLAS User Analysis

Although it has been suspected for some time that user analysis activities would present a different load pattern to production, until recently there has been surprisingly little work done in trying to quantify these differences. Indeed, if the LHC had begun operation on time, we would have been taking user analysis jobs without any solid knowledge of how they would perform. The delayed operation of the LHC has allowed the ATLAS VO to begin a campaign of automated tests against all Tiers of the WLCG infrastructure in order to begin such quantification work. Leveraging the Ganga Python-scriptable grid user interface framework, the HammerCloud tests submit large numbers of a "typical'' ATLAS user analysis job to selected sites, allowing load representative of that from a functioning LHC to be presented on demand.

In general, it is found that the ATLAS analysis workload stresses storage elements much more significantly than production work can. Although the ATLAS VO is in the process of adapting its workflow to reduce such load, it is clear that provisioning of site infrastructure on the basis of production workflow has resulted in many sites, including those in ScotGrid, being significantly underpowered for optimal analysis performance.

An analysis of the performance of Glasgow under HammerCloud tests, including optimisation work undertaken is presented our other paper in this proceedings. Durham has also begun testing the performance of their infrastructure, revealing similar problems to those at Glasgow before tuning was begun. Whilst 2 cores of an Intel Xeon processor are more than sufficient to support load on DPM services from production use, the extreme frequency of get requests generated by analysis jobs results in the node becoming saturated and effective performance bottlenecked.

# Local Users

As well as provision for WLCG/EGEE users, ScotGrid sites also have user communities local to each site. In this section, we will provide a brief overview of how these local users are provisioned and supported at Glasgow.

## Glasgow "Tier-2.5"

One of the largest groups of local users at Glasgow are the local particle physicists themselves. Whilst these users are all members of WLCG VOs and could simply submit jobs via the high-level submission frameworks, it is useful for them to be able to access other local resources when choosing to run jobs at the Glasgow site. This mode of operation, merging "personal" resources (often informally regarded as "Tier 3" of the WLCG) and Tier-2 compute provision is referred to as "Tier-2.5" provision within Glasgow.

The implementation mostly concerns tweaks to user mapping and access control lists, so that local users can be mapped from their certificate DN to their University of Glasgow username, rather than a generic pool account, whilst mapping their unix group to the same group as a generic VO member. This process involves some delicate manipulation of the LCMAPS credential mapping system: LCMAPS will only allow a given method of user mapping to be specified once in its configuration file; however, the Tier-2.5 mappings uses the same method (mappings from predefined list) as the default mapping method. The solution is to make a copy of the mapping plugin required, and call the copy for the Tier-2.5 mappings.

The resultant combination of user and group memberships allows Tier-2.5 users to access local University resources transparently, whilst also allowing them to access their WLCG-level resources and software. The provision of a local gLite User Interface service with transparent (ssh key) based login for local Physics users also helps to streamline access.

Similarly, the Glasgow site's storage elements allow local access to ATLAS datasets stored on them via the rfio and xroot protocols, through the use of their grid credentials whilst mapped to their local accounts. As the capacity of the Glasgow storage system is significant, this allows large data collections to be easily cached locally by researchers without the need for external storage.

## Glasgow NanoCMOS

The other significant user community at Glasgow are represented by the "NanoCMOS" VO, which represents electrical engineers engaged in simulation of CMOS devices at multiple levels of detail (from atomistic simulations to simulations of whole processors). As the only supported means of submitting to the Glasgow cluster is via the gLite interfaces, local NanoCMOS members currently use Ganga to submit jobs to the Glasgow WMS instances.

The typical NanoCMOS workload involves parameter sweeps across a single simulation, in order to generate characterisation of I/V curves and other data. This means that a single user will submit hundreds of subjobs to the WMS, which has challenged the scalability of the existing infrastructure. (Compare this to the recommendation that batches of jobs submitted to a gLite WMS should be of order 50 to 100 subjobs). Even the lcg-CE service has scalability problems in receiving on the order of 1000 jobs in a small time period; whilst it can manage thousands of concurrent runningjobs, the load on the CE is significantly higher during job submission and job completion, leading to component processes sometimes zombifying under the stress of massively bulky submission. Educating NanoCMOS users to stagger job submissions mitigates their effect on the system load, but this is an on-going problem. We are currently exploring other CE solutions, both the CREAM-CE and the NorduGrid ARC-CE, in order to examine their performance under similar load.

# Conclusions

This paper has presented an overview of the configuration and management decisions made at the ScotGrid distributed Tier-2. We have discussed the ways in which two of our component sites deviate from the "traditional" configuration of a WLCG site; both of which are likely to become mainstream given time.

In general, we have found that means of enhancing communication between the component sites are the most effective things to implement. The "virtual control room", in particular, has been of signal benefit to all of the sites in organising effort and transferring expertise in real-time. There is also an apparently unavoidable tension between the demands of WLCG use and those of local users, both for a traditional site with wholly-gLite mediated interfaces like Glasgow, and for a highly non-traditional site with a bias towards low-level access like Edinburgh/ECDF. LHCb and ATLAS statistics demonstrate that we are very successful Tier-2 from the perspective of Monte-Carlo production, which makes the difficulties we have had with simulated User Analysis load particularly notable (although much of this discussion is outside of the scope of this paper). It is likely that this will be our most significant issue in the coming year, although work is continuing to address this challenge.

# Acknowledgements

This work was supported by the GridPP project, funded by the UK Science and Technologies Facilities Council.

# References


abstract: Normalizing unwanted color variations due to differences in staining processes and scanner responses has been shown to aid machine learning in computational pathology. Of the several popular techniques for color normalization, structure preserving color normalization (SPCN) is well-motivated, convincingly tested, and published with its code base. However, SPCN makes occasional errors in color basis estimation leading to artifacts such as swapping the color basis vectors between stains or giving a colored tinge to the background with no tissue. We made several algorithmic improvements to remove these artifacts. Additionally, the original SPCN code is not readily usable on gigapixel whole slide images (WSIs) due to long run times, use of proprietary software platform and libraries, and its inability to automatically handle WSIs. We completely rewrote the software such that it can automatically handle images of any size in popular WSI formats. Our software utilizes GPU-acceleration and open-source libraries that are becoming ubiquitous with the advent of deep learning. We also made several other small improvements and achieved a multifold overall speedup on gigapixel images. Our algorithm and software is usable right out-of-the-box by the computational pathology community.
author: Goutham Ramakrishnan[^1]
Department of Electrical Engineering
Indian Institute of Technology Bombay
Mumbai, India
`email@example.com`
Deepak Anand
Department of Electrical Engineering
Indian Institute of Technology Bombay
Mumbai, India
`email@example.com`
Amit Sethi
Department of Electrical Engineering
Indian Institute of Technology Bombay
Mumbai, India
`firstname.lastname@example.com`
bibliography: references.bib
title: Fast GPU-Enabled Color Normalization for Digital Pathology

# Introduction

Tissue samples stained with any stain in general, and hematoxylin and eosin (H&E) in particular, suffer from variability in their appearances (see Figure ), which arise due to differences in the staining protocols and reagents used to process them. In digital pathology, the sensor response of the scanners used to capture their images can also add to this variability. While human visual perception automatically adjusts to differences in stain appearances, the performance of machine learning and deep learning algorithms that analyze these images depends on seeing enough variation in the training data. Conversely, the performance of these algorithms improves with color normalization . This makes color normalization a necessity when the data is sourced from only a few labs.

Several color normalization algorithms for histological images have been recently proposed, of which Khan *et al.*, Macenko *et al.* and Reinhard *et al.* are the most popular. Vahadane *et al.* proposed a technique for stain separation and color normalization called structure preserving color normalization (SPCN), and released its source code . They demonstrated that it performed qualitatively better in preserving biological structure, and quantitatively better in preserving stain densities compared to the previously popular techniques. However, no software exists for public use that can handle large gigapixel images that are common in digital pathology. Additionally, as well-motivated as SPCN is, it occasionally produces undesirable artifacts.

In this work, we significantly improve upon SPCN and its prior implementation to introduce a software for color normalization that (a) is free for non-commercial public use based on open-source software, (b) can handle large images without the need for embedding it in custom code to break up images, (c) scales gracefully in its runtime with image size by utilizing GPU acceleration, and (d) avoids undesirable artifacts that are common in the results of SPCN.

# SPCN Algorithm and its Shortcomings

We chose to improve and scale up the SPCN algorithm because it is intuitively satisfying in how it handles histological images. It first breaks a source image down into its stain densities, and then normalizes the color basis for each stain with respect to those of a target image. Additionally, the processes of stain density estimation and color normalization themselves are well-motivated. In this section, we revisit SPCN and document the strengths and shortcomings of its core algorithm and Matlab^^ implementation.

## SPCN Algorithm

Stain separation process in SPCN is based on the following insights about the staining process : (a) stain densities that modulate optical densities are non-negative, (b) their mixing proportions and color components are also non-negative, and (c) each pixel is likely to represent a spatially contiguous biological structure that absorbs largely one stain.

The major steps of SPCN based on these insights are given below :

1.  **Optical density computation:** The insight about the need to model the optical densities as opposed to the pixel intensities motivated the use of Beer-Lambert transform. Relative optical density $v_{c,x,s}$ corresponding to intensity $i_{c,x,s}$ for each channel $c \in \{{red,green,blue}\}$ for each pixel location $x$ for each image $s \in \{{source, target}\}$ is computed by assuming a maximum intensity $i_0$ (fixed to 255 for 8-bit images in the original implementation) corresponding to zero optical density and by using Beer-Lambert law as follows: $$v_{c,x,s} = \log\left(\frac{i_0}{i_{c,x,s}}\right)
    \label{eq:BLT}$$

2.  **Unsupervised stain density estimation:** The non-negativity of optical densities, stain densities, and their mixing proportions motivated the use of non-negative matrix factorization (NMF). Stain specificity of each biological structure motivates the use of sparse NMF (SNMF) to perform soft clustering of each pixel into stain clusters. The relative optical density $V_s$ (which is a matrix of $v_{c,x,s}$) can be decomposed into its constituent non-negative factors – the color basis matrix $W_s$ and the stain density matrix $H_s$ such that $V_{s} \approx W_{s}H_{s}$. To obtain this decomposition we solve the following optimization problem assuming that the rank $r$ of $H_s$ is two (number of stains) for H&E: $$\underset{W_{s},H_{s}}{\min} \: \|V_{s}-W_{s}H_{s}\|_{F}^{2} +  \lambda \displaystyle\sum_{j=1}^{r}\|H_{s}(j,:)\|_{1} \:, \quad\: W_{s}\geq0,H_{s}\geq0, \: \|W_{s}(:,j)\|_{2}^{2}=1
    \label{eq:Optimize}$$ The penalty on the $L1$ of $H_{s}$ induces sparsity, which is controlled by the hyperparameter $\lambda\geq0$ (fixed to 0.1 as per ). Dictionary learning is used to estimate $W_{s}$ and its pseudo inverse is used to compute $H_{s}$.

3.  **Color normalization:** After stain separation the source image is normalized with respect to the target image by replacing the former's color basis with that of the latter while preserving the former's relative stain densities. The source stain density for each stain is linearly scaled so that its $99^{th}$-percentile value matches that of the corresponding target stain. That is, the normalized matrix $V'_{\text{source}}$ is represented as: $$V'_{\text{source}} = \frac{P_{99}(H_{\text{target}})}{P_{99}(H_{\text{source}})} W_{\text{target}} H_{\text{source}}
    \label{eq:Normalize}$$

4.  **Normalized pixel intensity computation:** The source optical densities for each channel and each pixel thus normalized with respect to the target image are then converted back to the pixel intensity space using the inverse Beer-Lambert transform as follows: $$i'_{c,x,\text{source}} = i_0 \exp(-v'_{c,x,\text{source}})
    \label{eq:InvBLT}$$

## Occasional qualitative defects produced by SPCN

In our experimentation with SPCN, we observed the following problems:

1.  **Invalid maximum channel intensity assumption:** SPCN assumes a fixed maximum intensity value $i_0$ for each color channel in equations  and . When the background area of a slide has a color tinge or is slightly darker than the maximum possible brightness (e.g. 255), this assumption is no longer valid. This occasionally leads to undesirable color artifacts in the normalized image.

2.  **Inconsistency in extracting stain color prototype:** When there is lack of density variation in one stain due to a few very dark nuclei or a large proportion of light background, the optimization problem in equation  can become ill-conditioned. This problem is exacerbated in SPCN as it cannot process large images efficiently.

3.  **Unintentional stain swapping:** While solving the optimization problem in equation , the order of the two color bases (columns of $W_s$) and consequently stain density estimates (rows of $H_s$) can get swapped between the two stains.

## Challenges in applying SPCN to WSIs

To its credit, a fast approximate scheme for estimating $W_s$ in equation  was proposed for SPCN in , but it is still not a software that can be used out-of-the-box for WSIs. The Matlab^^ implementation of SPCN does not have support to read WSI formats such as $.svs$, $.ndpi$ (from popular scanner brands such as Aperio and Hamamatsu) to begin with, which is a platform integration issue. Additionally, it reads the entire source or target image in one go for estimating $H_s$ in step 2, which leads to program crashes for WSIs that cannot fit in the RAM. It does not utilize GPUs for computationally heavy tasks such as computing Beer-Lambert transform, its inverse, or normalizing each pixel in equations , ,and  respectively, even though GPUs are becoming commonplace due to the increasing application of deep learning to pathology. Using SPCN in a loop over patches in a compatible format extracted from a WSI is problematic, because it will simply normalize each patch independently, leading to color inconsistency over a WSI.

# Proposed Algorithm and its Implementation

We now describe key features of our software[^2].

## Improving qualitative results

We solved the problems mentioned in Section  as follows:

1.  **Neutral background color:** To fix the background tinge, we replaced the fixed maximum $i_0$ of equations  and  that deal with Beer-Lambert transform by channel-wise and image-wise maximum $\max_{x}(i_{c,x,s})$. For outlier rejection while maintaining efficiency for WSIs, we sample at most 100,000 pixels with intensity above 220, and choose their $80^{th}$-percentile as our estimate for the maximum channel intensities.

2.  **Color consistency over a WSI:** Although we seamlessly process WSIs of any size patch-wise, we estimate the color basis only once per WSI to maintain color consistency. This was not possible with SPCN because it can only treat patches of a WSI independently. We use large enough samples of non-white pixels (intensity less than 220) from the entire WSI to increase the variance of different stains observed, which prevents equation  from becoming ill-conditioned while estimating $W_s$. At the same time, because we sub-sample a gigapixel image, we solve the optimization problem efficiently. Additionally, while scaling using the $99^{th}$-percentile values attempts to reject dark outlier pixels while normalizing stain densities, this can still lead to problems in images with a large background portions. We use the $99^{th}$-percentile of the sub-sampled non-white pixels to reject problems of both dark outliers and white pixels.

3.  **Unintentional stain swapping:** The original heuristic implemented to get the stain order correct only compared the average blue channel of each stain. This leads to problems when the epithelium is light in color or the stroma also has a blue tinge. We resolved this issue by comparing the difference between red and blue channels across the two stains to get hematoxylin and eosin in the correct order.

## Improving efficiency and WSI compatibility

Our next set of contributions are in the form of a complete re-implementation of SPCN for efficiency and scale up to handle WSIs, as noted below:

1.  **WSI format compatibility using open-source software:** We implemented SPCN in python, which allowed us to use OpenSlide library for reading popular WSI file formats such as $.svs$ and $.ndpi$.

2.  **Efficient estimation of color basis:** As mentioned in Section , computation of $W_s$ can easily be scaled up by limiting the number of pixels used for stain color basis estimation, for which we use the open-source SPAMS library. Our software attempts to find a sufficient number of randomly scattered non-white pixels by finding at most 20 non-background patches on which the optimization problem of equation  is solved. This step thus scales sub-linearly with WSI size and is similar to the speedup scheme suggested in . It isn't constant because the search space for a sufficient number of pixels goes up with image size.

3.  **Handling gigapixel WSI through divide and conquer:** As mentioned in Section , computation of $W_s$ can easily be scaled up by limiting the number of pixels used for stain color basis estimation. On the other hand, steps that necessarily involve all pixels of the source image, e.g. computation of Beer-Lambert transform and inverse and the estimation of the normalized optical densities in equations , , and  were performed serially on patches of the whole WSI. This required time consuming disk reads and writes, which were marginally sped up by sampling longitudinal patches from a WSI instead of the usual square patches based on the insight about row-wise contiguous storage of a WSI on a hard drive.

4.  **Efficient computation of order statistics:** As mentioned previously, some steps required computation of a $99^{th}$-percentile of stain density for outlier rejection, which we efficiently implemented for the WSI by taking medians of that percentile across patches.

5.  **Speedup using TensorFlow:** First, re-implementation in python instead of Matlab itself led to a significant speedup for small images. Then, we implemented the heavy lifting computations of the algorithm in TensorFlow , which easily deploys any available CPU or GPU cores for speed.

6.  **Software functionalities and easy debugging:** The new software is modular, easy to understand and user-friendly. The ability to normalize a batch of images using the same target image is also available for user efficiency, and so is a demo and a verbose mode for better understanding.

# Results

We present an evaluation of the quality of the output normalized image of our improved SPCN algorithm compared to the original SPCN, and discuss the speed of our implementation on varying image sizes.

## Quality of Color Normalization

A qualitative and quantitative comparison of the SPCN algorithm with other color normalization techniques is described in , and justifies its choice as a starting point in this work. In Figure , we compare the performance of the original SPCN to our improved version on three challenging examples of source images taken from TCGA. On Image (a), the original SPCN exhibits swapping of stain color basis due to the presence of significant blue components in both stains. Our proposed improvement to compare the difference of red and blue components avoids this error. Image (b) illustrates that in cases where one stain dominates, SPCN can lead to a color tint in the intermediate whitespace. Our algorithm avoids this by estimating a channel-wise maximum pixel intensity. Image (c) shows an image with a significant background portion, which leads to a strong tinge in the background after normalization in an extreme case, which is also handled well in our results by estimating the maximum intensity for each channel separately.

```latex
\begin{figure*}[ht]\begin{minipage}{ \textwidth}
  \begin{tabular}
     {p{0.04\textwidth}p{0.31\textwidth}p{0.31\textwidth}p{0.31\textwidth}}
%      \hline
      & \hspace{15pt} Original Image & \hspace{14pt} Original SPCN~\cite{TMICN17} & \hspace{12pt} Improved SPCN \\
%      \hline

%       \vspace{28pt} 1 &
%       \vspace{2pt} \parbox[c]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_blueoverstain_b032}} &
%       \vspace{2pt} \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_blueoverstain_wrong}} &
%       \vspace{2pt} \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_blueoverstain_correct}} \\

%      \vspace{28pt} 2 &
       (a) &
%      \vspace{2pt} \parbox[c]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_bluewhite_is063}} &
      \parbox[c]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_bluewhitecrop_is063}} &
       \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_bluewhitecrop_wrong}} &
       \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_bluewhitecrop_correct}} \\

      (b) &
      \parbox[c]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_redoverstaincrop_A8-A06Z}} &
       \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_redoverstaincrop_wrong}} &
       \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_redoverstaincrop_correct}} \\

       (c) &
       \parbox[c]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_redwhitecrop_B_2_5}} &
       \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_redwhitecrop_wrong}} &
       \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_redwhitecrop_correct}} \\

%       \vspace{28pt} 5 &
%       \vspace{2pt} \parbox[c]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_mostlywhite_A_5_6}} &
%       \vspace{2pt} \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_mostlywhite_wrong}} &
%       \vspace{2pt} \parbox[l]{0.31\textwidth}{ \includegraphics[width=1.42in]{img_table_mostlywhite_correct}} \\

%      \multicolumn{4}{c}{All of the above images have been normalized using Figure \ref{target}.} \\
%      \hline
  \end{tabular}
  %\vspace*{1pt}
  \caption{Comparison of image quality between original and improved SPCN.}
  %\vspace*{-10pt}
  \label{fig:results}
  \end{minipage}
\end{figure*}
```

## Runtime analysis on gigapixel images

The flowchart in Figure summarizes the steps of the algorithm, highlighting the efficiency achieved by our software through GPU computations. Even without GPU's, the TensorFlow implementation is much faster than the original Matlab implementation. A step-wise comparison of the time taken for color normalization by the almost original SPCN (modified to accommodate large images using python and NumPy) and our TensorFlow implementation using CPU and GPU respectively is shown in Figure . For relatively smaller images, the speeds of the implementations are comparable. Additionally, the time taken to estimate $W_s$ remains almost constant for large image sizes due to our efficient implementation. As the images become very large, even though other steps scale linearly with image size, our algorithm achieves a multi-fold speedup over a simple python implementation. It takes only a few minutes for even 5 gigapixel images.[^3] This shows that it is practical to use this software on large pathology images right out-of-the-box.

## Comparison with other Deep learning methods

SPCN has been well validated against the most commonly used algorithms in the base paper . We compare the presented work to most recent literature , and .These papers presents a Generative Adversarial Network (GAN) for color normalization. In the proposed method requires retraining once the target or reference image changes. In , proposed method do not require the retraining when the reference image changes. In algorithm also presents many advantages over \[BenTaieb\] (which was published in March 2018). Firstly, the stain transfer procedure using GANs is a completely different approach to color normalization and it is a domain which is still largely unexplored and does not guarantee theoretical performance. Secondly, the algorithm does not perfectly preserve biological structure as it only approximates it using a loss function. Finally, it involves training of deep neural networks, requiring availability of training data, heavy computations, adjustment of hyperparameters and application to large images intractable. Moreover, it presents a black-box to the internal computations involved, making debugging difficult. Our algorithm in comparison, is well-validated, preserves structure, is simple to understand and debug, requires simple computations and with our software, can be used on images of any size. An additional advantage of our algorithm is that it contains a stain separation step as an integrated module, which can aid research work involving any particular stain in the histological images.

# Conclusion

Uptake of computational pathology suffers from two major problems – lack of investment in whole slide scanners, and the lack of fast algorithms to process whole slide images. With this work, we have contributed to overcoming the second challenge by introducing a open-source software that can color normalize WSIs in a reasonable time. Our software significantly improved upon SPCN  in this respect, while also improving on the image quality and retaining the advantages of SPCN. With the advent of deep learning, several research groups are using GPUs and open-source software such as OpenSlide  and TensorFlow  to work with WSIs. Our improved and corrected implementation of SPCN is built on top of such libraries. To ensure that working with large slides does not seem daunting to medical researchers, it is necessary to make other widely-used parts of a computational pathology pipelines, such as nucleus detection and segmentation  and gland segmentation, also scalable and open-source.

[^1]: Equal contributions by first two authors

[^2]: Software available at: <https://github.com/MEDAL-IITB/Fast_WSI_Color_Norm>

[^3]: Computer: CPU Core i7, 4.2GHz, 8 core, 64GB RAM; GPU Titan X 12GB RAM


author: P. H. Keys, M. Mathioudakis , D. B. Jess , D. H. Mackay; F. P. Keenan
date: Received November 2013; accepted May 2014
title: Dynamic properties of bright points in an active region

Bright points (BPs) are small-scale, magnetic features ubiquitous across the solar surface. Previously, we have observed and noted their properties for quiet Sun regions. Here, we determine the dynamic properties of BPs using simultaneous quiet Sun and active region data. The aim of this paper is to compare the properties of BPs in both active and quiet Sun regions and to determine any difference in the dynamics and general properties of BPs as a result of the varying magnetic activity within these two regions. High spatial and temporal resolution G-band observations of active region AR11372 were obtained with the Rapid Oscillations in the Solar Atmosphere instrument at the Dunn Solar Telescope. Three subfields of varying polarity and magnetic flux density were selected with the aid of magnetograms obtained from the Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory. Bright points within these subfields were subsequently tracked and analysed. It is found that BPs within active regions display attenuated velocity distributions with an average horizontal velocity of $\sim$`<!-- -->`{=html}0.6 km s$^{-1}$, compared to the quiet region which had an average velocity of 0.9 km s$^{-1}$. Active region BPs are also $\sim$`<!-- -->`{=html}21% larger than quiet region BPs and have longer average lifetimes ($\sim$`<!-- -->`{=html}132 s) than their quiet region counterparts (88 s). No preferential flow directions are observed within the active region subfields. The diffusion index ($\gamma$) is estimated at $\sim$`<!-- -->`{=html}1.2 for the three regions. We confirm that the dynamic properties of BPs arise predominately from convective motions. The presence of stronger field strengths within active regions is the likely reason behind the varying properties observed. We believe that larger amounts of magnetic flux will attenuate BP velocities by a combination of restricting motion within the intergranular lanes and by increasing the number of stagnation points produced by inhibited convection. Larger BPs are found in regions of higher magnetic flux density and we believe that lifetimes increase in active regions as the magnetic flux stabilises the BPs.

# Introduction

Bright points (BPs) are ubiquitous in the solar atmosphere. With field strengths of the order of a kilogauss, they are thought to be the foot-points of magnetic flux tubes that have formed as magnetic flux is advected from the centre of granules into inter-granular lanes . Reduced pressure within the flux tube and the hot bottom effect result in BPs appearing as localised intensity enhancements in the photosphere . Away from disk centre, the inclination of the partially evacuated flux tube allows the hot wall of the adjacent granule to be observed. Observations in the G-band (4305Å) are often used as a proxy for studying BPs as they appear brighter due to the reduced abundance of the CH molecule at higher temperatures .

The magnetic field enclosed within the boundary of a BP can act to guide waves in the solar atmosphere . Recent 3-D MHD models have investigated the effect of Alfvén waves generated by the motion of BPs. In addition to being linked to Alfvén wave generation, BPs have been employed to identify locations of photospheric vortices by observing rotational elements in their motions as they move towards downdraughts. They have also been observed in simulated studies with localised vortex flows found at the location of some simulated BPs . Observational work has also shown the propagation of waves through BPs , while other research suggests they are the source of spicule oscillations .

Several studies have focused on BP properties such as size, velocity distribution and peak intensity . In particular, investigated the transverse velocities of over 6000 BPs using G-band observations of a quiet Sun region and numerical simulations. The observed velocities were as high as 7 km s$^{-1}$, average velocities were 1 km s$^{-1}$, and the BP lifetime was around 90 s. This study was extended to the upper photosphere/lower chromosphere using the Ca  K core where delays were found in the motion of BPs between the two layers along with elevated velocities observed in Ca  K BPs. A subsequent, independent study of data by found similar elevated velocities in Ca  H BPs.

investigated quiet Sun and network BPs and found mean velocities of 1.1 km s$^{-1}$ and 0.95 km s$^{-1}$, respectively. The results were obtained with a temporal resolution of 25 s and a spatial resolution of 0$''$.83. studied both a decaying active region and a quiet Sun region with the Interferometric BIdimensional Spectrometer and found that the velocities follow a Gaussian distribution with values as high as 2 km s$^{-1}$, but did not highlight any differences between BPs found in an active region compared to a quiet region. Likewise, observed BPs in a sunspot moat and found a mean velocity of 1.11$\pm$`<!-- -->`{=html}0.7 km s$^{-1}$. They suggested that the velocity values are attenuated in an active region by 0.2 – 0.35 km s$^{-1}$ compared to the solar network.

Another important component related to BP dynamics is dispersion . The dispersion of magnetic elements is thought to be induced by photospheric flows . A scalar diffusion coefficient ($K$) and the diffusion index ($\gamma$) are frequently estimated to determine the efficiency and the mechanism of diffusion respectively. The diffusion index, $\gamma$, can be established by monitoring the squared displacement ($sd$) of each BP over each time-step ($\tau$) in the BPs lifetime and is given by the relation $sd \propto \tau^{\gamma}$. If $\gamma = 1$ the dispersion is defined as *normal diffusion* (i.e. a random walk) whereas when $\gamma < 1$ the process is described as *sub-diffusive* and when $\gamma > 1$ the process is described as *super-diffusive*.

Previous studies which use BPs as tracers in establishing this diffusion have found varying values for $K$ and $\gamma$ and therefore varying diffusive regimes. found a sub-diffusive regime in photospheric images of an active region with sub-diffusive motions also observed in a study by . However, reported that the sub-diffusive motions were found for timescales of 0.3–22 minutes, while normal diffusion occurs in the timescale of 25–57 minutes. More recent studies of photospheric BPs provide evidence of motions which are super-diffusive. also found super-diffusive motions in Ca  H images of the lower chromosphere/upper photosphere. It has been suggested that estimations of the diffusion index may be effected by the lifetimes of the magnetic elements used to calculate the value.

In this study we compare the dynamic properties of BPs such as size, lifetime, diffusivity and horizontal velocity distributions in simultaneously observed active and quiet Sun regions, using high spatial and temporal resolution observations of the photosphere.

# Observations & Data Reduction

Observations were obtained using the Rapid Oscillations in the Solar Atmosphere instrument at the 76 cm Dunn Solar Telescope (DST) in New Mexico, USA. Data were acquired on 2011 December 10 under excellent seeing conditions that remained consistent for approximately 2 hours. The target was AR 11372, located at N7.6W4.2 in the conventional heliographic co-ordinate system. For this study, a 9.2 Å wide filter centred at 4305 Å (G-band) was employed to study a 70$''\times$`<!-- -->`{=html}70$''$ field-of-view (Figure ) with a spatial scale of 0$''$.069 pixel$^{-1}$. Speckle reconstruction algorithms were employed utilising bursts of 64 images for a single speckle reconstruction, resulting in a post reconstruction cadence of 2.112 s. Image destretching was performed using a 40$\times$`<!-- -->`{=html}40 grid .

Line-of-sight magnetograms for the active region were obtained with the Helioseismic and Magnetic Imager on-board the Solar Dynamics Observatory . The instrument was employed with a cadence of 45 s and a two-pixel resolution of 1$''$.0. Additionally, one contextual HMI continuum image was obtained to aid the co-alignment of magnetograms with those from the lower solar atmosphere. The HMI data was processed with the standard `hmi_prep` routine, including the removal of energetic particle hits. Subsequently, subfields were chosen from the processed data, with a central pointing similar to that of the ground-based image sequences. The HMI continuum image was used to define the absolute solar co-ordinates before the ground-based observations were subjected to cross-correlation techniques to provide sub-pixel co-alignment accuracy between the two imaging sequences. Following co-alignment, the maximum $x$- and $y$-displacements were both less than one tenth of a HMI pixel, or 0$''$.05 ($\approx$`<!-- -->`{=html}38 km). A sample of a co-aligned ROSA G-band image and a HMI magnetogram is shown in Figure .

The spatial resolution of the reconstructed G-band images was estimated using methods outlined by (Appendix A) and . A spatial resolution of around 0$''$.17 was established across a range of images in the dataset. This value was found to be roughly constant throughout and is close to the theoretical diffraction limit (0$''$.14) for the telescope at this wavelength.

# Data Analysis & Results

We selected three areas of equal dimensions (15.5$''\times$`<!-- -->`{=html}20$''$) from the G-band data for further analysis (Figure  (a)). These regions were selected based on a combination of the relative density of BPs in the region, and their position with respect to the polarity of the active region as determined from co-aligned HMI images (Figure  (b)). One of the regions lies between two pores of the same polarity (average $\Phi$ = 169 G; average $| \Phi |$ = 169 G; Figure  (c): 1), the second in a relatively quiet region in the field-of-view (average $\Phi$ = 3 G; average $| \Phi |$ = 7 G; Figure  (c): 2) and the third in a region of opposite polarity to the first region, with reduced magnetic flux (average $\Phi$ = -101 G; average $| \Phi |$ = 107 G; Figure  (c): 3). It must be stressed that the dimensions of all three regions were kept equal so that a meaningful comparison can be carried out. All results are summarised in Table .

Once the three regions were selected, the BPs in each were tracked using the algorithm employed by . Some of the most important aspects of this code will be discussed here, but for a full discussion see and for improvements and additions implemented see and .

The code has two main procedures; detection and tracking. The detection part of the code maps out the intergranular lanes and a binary map is produced of both the granules and the BPs. A compass search is then combined with intensity thresholding to remove granules from the resulting binary map. Because of the sharp turning points present at BP boundaries, this method allows an accurate determination of the boundaries. To avoid spurious detections of noise, the minimum threshold size for a BP to be detected is set at 4 pixels ($\approx$`<!-- -->`{=html}10 000 km$^2$). demonstrated that this is a suitable lower limit, with BP areas found to follow a log-normal distribution with a peak significantly above this limit. To avoid the detection of fragmenting granules, the upper limit for BP area is set at 200 pixels ($\approx$`<!-- -->`{=html}500 000 km$^2$). This limit is sufficiently high to avoid excluding larger BP chains.

```latex
\begin{table*}[t!]\centering
\caption{Summary of BP characteristics in AR\,11372.}
\label{keys_table_1}
\scalebox{1.0}{
%\begin{tabular}{c *{4}{c}}
\begin{tabular}{l c c c}
\hline\hline
{\textbf{Region}} & {\textbf{1}} & {\textbf{2}} & {\textbf{3}} \\
\hline
{\textbf{No. of BPs}} & 2755 & 851 & 2843 \\
{\textbf{Av. $\Phi$ (G)}} & 169 & 3 & -105\\
{\textbf{Av. $| \Phi |$ (G)}} & 169 & 7 & 107\\
{\textbf{Av. Lifetime (s)}} & 136$\pm$40 & 88$\pm$23 & 128$\pm$38 \\
{\textbf{Av. Area (km$^2$)}} & 29\,000$\pm$2{\,}600 & 24\,000$\pm$2{\,}100 & 28\,000$\pm$2{\,}500 \\
{\textbf{Av. Velocity (km\,s$^{-1}$)}} & 0.6$\pm$0.3 & 0.9$\pm$0.4 & 0.7$\pm$0.4 \\
{\textbf{Max. Velocity (km\,s$^{-1}$)}} & 5 & 7 & 5.3 \\
{\textbf{Diffusion Index ($\gamma$)}} & 1.23$\pm$0.22 & 1.21$\pm$0.25 & 1.19$\pm$0.24\\
{\textbf{Diffusion Coefficient ($K$; km$^2$\,s$^{-1}$)}} & 109$\pm$43 & 123$\pm$45 & 137$\pm$56 \\
\hline
    \end{tabular}
    }
\end{table*}
```

Once possible BP features are detected, the resulting binary map of detected features is passed through to the tracking algorithm to assign each detected feature with a tracking number and to establish essential properties such as area and the BP barycentre. For each BP detected, the algorithm searches an area around the detected BP in subsequent frames to find associated features. If another object falls within the search area, then the two features are considered as the same object and treated as such. If nothing is seen in the search area in subsequent frames, the object is considered to no longer exist. For cases in which a feature is missed in a frame before being found again in the next frame, e.g. due to a slight drop in seeing conditions in the data, the binary for the feature prior to being lost and immediately after being discovered again is used to determine the location of the missed feature. This ensures that the feature is tracked across all frames. Important information related to each feature, such as the area and barycentre for each frame, is then recorded for future use. For this study we ignore any BP that is found to originate from or found to pass beyond the boundary of our three selected regions. This alleviates certain issues that may arise in such scenarios, particularly with regards to the lifetimes of these features.

The average horizontal velocity established for the first region in Figure  (hereafter region 1) was 0.6 km s$^{-1}$ from the 2755 BPs observed. For this region the highest velocity observed was 5 km s$^{-1}$, the average BP lifetime was 136 s and with this the longest-living BP existed for $\sim$`<!-- -->`{=html}50 minutes. In the second region (hereafter region 2) we tracked 851 BPs which displayed remarkably similar characteristics to the quiet Sun results of . The average horizontal velocity for this region was found to be 0.9 km s$^{-1}$, with the highest velocity measured being $\sim$`<!-- -->`{=html}7 km s$^{-1}$. The corresponding average lifetime is 88 s and the longest-lived BP was observed for $\sim$`<!-- -->`{=html}15 minutes. Our third area was located between two regions of opposite polarity within AR 11372 (hereafter region 3) and its BPs had characteristics similar to region 1. The average horizontal velocity of the 2843 BPs detected in this region was 0.7 km s$^{-1}$, maximum velocity 5.3 km s$^{-1}$, average BP lifetime 128 s and the maximum lifetime approximately 19.5 minutes. In Figure  we show the corresponding velocity distributions for each region.

Our results highlight that BPs in active regions exist longer than their quiet Sun counterparts, with lifetimes of both region 1 (136 s) and region 3 (128 s) longer than region 2 (88 s).

The areas for the BPs were observed to follow a log-normal distribution (Figure ). This semi-log plot reveals peaks in region 1, 2, and 3 at roughly 29 000 km$^2$, 24 000 km$^2$, and 28 000 km$^2$, respectively. This gives a $\sim$`<!-- -->`{=html}21% increase in size from the quiet to the more active regions. The size of a BP is often reported as the diameter that one would have if assumed to be a circle. With this, the peak diameter for the three subfields considered in this study are 190 km, 175 km, and 189 km for regions 1, 2, and 3, respectively. The number of BPs detected in both active subfields is 0.38 BPs per Mm$^2$, compared to only 0.09 BPs per Mm$^2$ in the quiet region.

Given the ability of the algorithm to track large numbers of BPs, we decided to investigate if the BPs experience any preferential flow directions in their motions within each subfield. We examined the motions in relation to the centre-point of each of the three subfields and find no overall preferential flow direction (Figure ). In region 1, where the centre-point is between 2 pores of the same polarity, 43% of the detected BPs move away from the centre of the dataset, about 48% move towards the centre, while the remaining 9% experience no change in direction. The average velocity of those BPs moving towards or away from the centre point is 1.1 km s$^{-1}$ and 1 km s$^{-1}$, respectively. Hence there is no real difference in the velocities of BPs moving towards or away from the centre of the subfield. The values are however slightly elevated in comparison to the average for the entire region (an overall average velocity of 0.6 km s$^{-1}$ was established for this dataset). This is expected due to the fact that a BP moving towards or away from the centre point will have some velocity component. By contrast, the overall distribution will also contain stationary BPs, which will lower the mean horizontal velocity component observed for the region as a whole. In region 2 it was observed that 50% move away from the centre and 45% move towards it, with the remaining 5% experiencing no change in direction. Again, the average velocity for both moving towards and away from the centre point is slightly elevated at 1.5 km s$^{-1}$ and 1.4 km s$^{-1}$, respectively, when compared to the average velocity for the region. Within region 3 there is no preferential flow direction from the centre of the subfield, with 46% moving away from the centre point, 46% moving towards it and 8% having no preferential direction. Velocities are similar to those found in region 1.

Additionally, we studied motions about the neutral line between two pores of opposite polarity in AR 11372[^1]. The neutral line was found using LOS magnetograms of region 3 and was employed as a point of reference to establish the movements between these two pores. About 51% of the BPs in the field-of-view move away from the LOS null-point and 41% move towards it. The velocity values for BPs moving away from the neutral line are 1.1 km s$^{-1}$, compared to 1.3 km s$^{-1}$ for those moving towards it. Figure  shows that there is some evidence of flows between quadrants in each subfield, however, more studies need to be carried out on various active region data sets to verify if these are real aspects of BP dynamics or anomalies.

We estimate the diffusion coefficient, $K$, and the diffusion index, $\gamma$, for the three regions. This is performed by monitoring the squared-displacement ($sd$) for each tracked BP at each time step ($\tau$) in their lifetime . We establish $\gamma$ for each BP by plotting $sd$ as a function of $\tau$ on a log-log scale. We find $\gamma$ with the gradient of the plot as the two variables are related by the equation: $$sd(\tau) = C \tau^{\gamma},$$ where $C$ is a constant of proportionality. As a value of $C$ can be evaluated from the log-log plot ($C = 10^{\gamma_{intercept}}$), we then establish the diffusion coefficient with the equation: $$K(\tau) = \frac{C \gamma}{4}\tau^{\gamma - 1}.$$

This method finds values for $\gamma$ in region 1, 2, and 3 as 1.23$\pm$`<!-- -->`{=html}0.22, 1.21$\pm$`<!-- -->`{=html}0.25, and 1.19$\pm$`<!-- -->`{=html}0.24, respectively. This suggests normal diffusion to slightly super-diffusive motions in the BPs in all three subfields. The corresponding diffusion coefficients for the three regions are 109$\pm$`<!-- -->`{=html}43 km$^2$ s$^{-1}$, 123$\pm$`<!-- -->`{=html}45 km$^2$ s$^{-1}$, and 137$\pm$`<!-- -->`{=html}56 km$^2$ s$^{-1}$ respectively. The time ranges for the these estimates of the diffusion coefficients are $\sim$`<!-- -->`{=html}0.5–50 minutes, $\sim$`<!-- -->`{=html}0.5–15.5 minutes, and $\sim$`<!-- -->`{=html}0.5–19.5 minutes respectively.

# Discussion

## Horizontal Velocity Properties and Lifetimes

The histograms showing the velocity distributions for the three regions (Figure ) reveal, as do the average and maximum transverse velocity values, that BPs within areas of stronger magnetic field appear to have less dynamic behaviour. BPs with higher velocities ($>$`<!-- -->`{=html}2 km s$^{-1}$) have been linked to the production of magneto-sonic kink waves . Region 2 has the highest fraction of BPs with velocities $>$`<!-- -->`{=html}2 km s$^{-1}$, with about 15% of the BPs studied having velocities greater than this value. This is higher than in the two more-active regions, where the number of BPs with velocities greater than 2 km s$^{-1}$ in region 1 and 3 was estimated at 9% and 12%, respectively. It is expected that the higher field strengths within the active regions will inhibit convection by suppressing chaotic flows , which will then act to reduce the velocity distribution of BPs within regions of higher magnetic flux density, as is observed in the present study. This could explain the slightly lower horizontal velocities observed in region 1, as the HMI magnetograms show that the maximum flux density is higher in this region ($\approx$`<!-- -->`{=html}1700 G) compared to region 3 ($\approx$`<!-- -->`{=html}1100 G).

Our transverse velocities compare well with studies of BPs around active regions. Most of these find mean horizontal velocities for BPs in active regions in the range 1 – 2 km s$^{-1}$, similar to those found here. It is also noteworthy that some observations find that BP velocities are attenuated in active regions compared to quiet Sun regions, as in the present work. Indeed find that velocities in active regions are attenuated by about 0.2 – 0.35 km s$^{-1}$, while in our study the velocities between the quiet Sun region and the two active region subfields are also attenuated by about 0.2 – 0.3 km s$^{-1}$.

Also of note are the results with regard to region 2. The horizontal velocity value of 0.9 km s$^{-1}$ found for the 851 BPs studied is remarkably similar to the quiet Sun disk centre study of , who observed over 6000 BPs and determined an average velocity of 1 km s$^{-1}$. These results, coupled with the HMI magnetograms provides confidence that the subfield of region 2 within the active region data presented here is indeed a quiet region. This, therefore, leads to a meaningful comparison between the quiet region and the two active region subfields. It should also be noted that the results presented here help improve our current understanding of the properties of BPs in active regions, as the dynamic properties are established simultaneously in quiet Sun and active region data.

With regards to BP lifetimes, the results for region 2 echo the 90 s average lifetime reported for the quiet Sun disk centre observations of . find similar lifetimes for quiet Sun BPs in their study. It should be noted that report a mean lifetime of 263$\pm$`<!-- -->`{=html}137 s for BPs observed in an active region which is longer than the value derived in the present study. However, the error bars are large in the case of . Lifetimes of BPs can be difficult to ascertain exactly. The main issue arises in determining when the BP first appears and then disappears. Therefore the method employed for tracking the BP can have an impact on the lifetime. For example, methods that require human input in the identification process will tend to have larger lifetime estimates as the observer may actively look for associated features in subsequent frames and may introduce human error by actively trying to associate erroneous bright features to the BP being tracked. With fully-automated algorithms this is less of an issue but there needs to be some method for the algorithm to search for associated features or else the lifetimes will be underestimated. The algorithm used in this study looks 30 s ahead for associated features. If something is found, then it is treated as a continuation of the original feature. If nothing is observed within that time frame, then the feature is determined to have disappeared. We believe that the longer lifetimes displayed in the active region subfields in this study are the result of a higher magnetic flux density within these regions stabilising the BPs compared to quiet regions.

## Area Distributions and BP Numbers

The area distributions shown in Figure  for the three regions follow a similar log-normal distribution to those found in previous studies . suggest that this distribution is the result of a fragmentation mechanism, similar to that found in sunspot umbrae . As well as having a peak at a larger value, the width of the peak is also larger for the active regions compared to the quiet one. Such a broader distribution with a larger peak between active region and quiet Sun data is also observed by . However, a study by suggests that there is little difference in both their active region and quiet Sun data with regards to the area distribution. Such differences can often be attributed to the resolution of the data and the feature tracking algorithm employed, specifically in this case, how the algorithm treats BP chains .

In terms of BP diameters, a study of G-band BPs in an active region by found a distribution of diameters between 100 – 300 km with the peak at 160 km. Similarly, used IBIS to scan Fe  in both a bi-polar active region (7090 Å scan) and a quiet Sun region (6302.5 Å scan) and estimated the size distribution of BPs in both regions. In particular, they observed a peak in the active region distribution at 326 km and one in the quiet region at 181 km. However, the authors note that the high value found in their active region distribution is likely the result of poor seeing conditions during the acquisition of the dataset and the lower spatial resolution of IBIS. find similar results on the effect of spatial resolution on BP size distributions. In their study, the photospheric level of a Na  D$_1$ IBIS scan observed a peak in BP size distributions at a diameter of 255 km, whereas ROSA G-band images had a peak in the distribution at a diameter of 200 km. Given that the data in the current paper are from three subfields within the one field-of-view, our results are more readily comparable as all three subfields will be subjected to the same seeing conditions over the duration of the observations.

The number of BPs found in the active region subfields (0.38 BPs per Mm$^2$) are again comparable to previous estimates of 0.3 BPs per Mm$^2$ and 0.34 BPs per Mm$^2$ in active regions. However, the value for region 2 is quite low in comparison to other studies. For example, 0.22[^2] BPs per Mm$^2$ and 0.19 BPs per Mm$^2$ . This is likely a consequence of the smaller field-of-view used to obtain the results, or perhaps the region selected is particularly quiet and far away from a network boundary. It should also be noted that recent estimates by indicate BP densities three times higher than those found previously, with a value of 0.97 BPs per Mm$^{2}$ reported. The authors attribute this difference to automated detection algorithms which they argue miss fainter objects. However, there is also an argument that over segmentation of longer BP chains is undesirable as it will increase the density of BPs reported. The next generation of solar telescopes and instruments will aid in improving these estimates of BP number densities and area distributions as they will allow a more accurate description of BP chains, and therefore, will allow us to determine if these chains can be segmented or not.

## Flow Directions and Diffusion

We looked at the motions of all BPs tracked to determine if preferential flow directions were present in their motions. The overall numbers moving away/towards the centre point in each region, with the possible exception of those about the location of the neutral line in region 3, show no particular preferential flow direction. We expanded on this by analysing the flow directions in each quadrant of the three regions about the centre-point and the neutral line found in region 3. The results in Figure  show that there are slightly elevated numbers flowing away or towards the reference point under investigation in some quadrants, for example, the polar area diagram for the motions with respect to the neutral line (Figure  (d)) suggests a greater number moving away from the neutral line in the NE quadrant (i.e. towards the pore near the centre of the data set). Given that only one active region was analysed in this study we cannot comment on whether such motions are a feature of the BPs dynamic properties with any degree of certainty. Also, with the lack of sufficient spatial resolution in regards to the magnetic field information for the BPs in this data set, we cannot formulate any theories or possible drivers that would explain these motions or if they are just random anomalies.

To aid in defining the reasons for the attenuation of velocities in region 1 and 3, we established the diffusion index ($\gamma$) and the diffusion coefficient ($K$) using the methods described by . Our results suggest that the diffusion mechanism in all three regions is the same and shows that the BPs have normal diffusion (i.e. a random walk) to slightly super-diffusive motions. This is can be seen because we find values for $\gamma$ in region 1, 2, and 3 as 1.23$\pm$`<!-- -->`{=html}0.22, 1.21$\pm$`<!-- -->`{=html}0.25, and 1.19$\pm$`<!-- -->`{=html}0.24, respectively.

The coefficient $K$ indicates the efficiency of the dispersion of the BPs as it measures the rate of increase in the dispersal area in units of time for all the BPs. Values for $K$ were found to be 109$\pm$`<!-- -->`{=html}43 km$^2$ s$^{-1}$, 123$\pm$`<!-- -->`{=html}45 km$^2$ s$^{-1}$, and 137$\pm$`<!-- -->`{=html}56 km$^2$ s$^{-1}$ in region 1, 2, and 3 respectively. The values of $K$ that we establish in the three regions indicate that the BPs in region 3 experience the quickest rate of diffusion while those in region 1 experience the slowest rate of diffusion of the three subfields. This may seem unusual, as one may expect regions with higher dynamic behaviour (such as region 2) to have a higher diffusion coefficient. However, the error bars on the value of $K$ estimated for region 3 are significantly larger than those for the other two regions.

These results, to an extent, are similar to previous studies. studied BPs in G-band and found values in the range 50–80 km$^2$ s$^{-1}$. When the authors reconstructed the velocity field using correlation tracking techniques, they employed artificial passive corks which showed diffusion coefficients of 77, 176, and 285 km$^2$ s$^{-1}$ in magnetic, network and quiet regions respectively. Likewise, find values of 110 and 250 km$^2$ s$^{-1}$ in the core of a plage region and surrounding quiet regions respectively.

The results from region 2 in this study (i.e. the quiet region) are at odds with previous results. As highlighted in previous studies , the temporal scales can effect the diffusion coefficient. report diffusivities of 70–90 km$^2$ s$^{-1}$ for timescales of less than three hours and values between 200–250 km$^2$ s$^{-1}$ for timescales longer than eight hours. As the timescale that the values for region 2 are estimated from are considerably short, this may influence the diffusion values we find. Also, similar to the estimation of BP lifetimes, the method for tracking the BPs may effect the diffusion coefficients estimated. Taking the example of manual feature tracking again, by over-estimating the lifetime, the squared displacement values will be higher and there may be more Lévy flights in the BP paths which would increase the estimates of $\gamma$ and $K$. Furthermore, as diffusion can occur at varying spatial scales , the total diffusion within the region under investigation is the resultant of the diffusivity at varying spatial scales (i.e. at granular and super-granular scales). This could lead to anomalies, however, we would only know for certain by expanding this study to numerous active region data sets.

The results with regards to the diffusion processes in these three regions do provide some important clues to the dynamic properties of BPs, however. The diffusion index established for the three regions shows that the same diffusion mechanism is found in both the quiet subfield and the active region subfields. This is important in that it shows that the motion of BPs is governed mainly by the buffeting motion of the granules within which these features are confined. This would suggest that the magnetic flux is passively tracing granular evolution. This process may explain the attenuation of BP velocities in active regions. In an active region where convection is inhibited by the magnetic field, there will be more stagnation points at the intersection of several granules. The motion of BPs in these stagnation points will be reduced and therefore will result in less dynamic behaviour and the attenuation of the velocities. Values for $K$ obtained previously at both quiet and active regions appear to support a greater number of stagnation points in active regions which would lead to reduced velocities compared to quiet regions.

# Conclusions

The work presented here has studied the differences in the properties of BP found in both active regions and quiet Sun regions using high spatial and temporal resolution observations of the photosphere obtained with ROSA. Three subfields were considered in the G-band observations of an active region: an inter-pore region, a quiet region and a negative polarity region. These were chosen using HMI magnetograms and selected for their varying magnetic environments.

The results suggest that the transverse velocity component of BPs is reduced ($\sim$`<!-- -->`{=html}0.6 km s$^{-1}$) within the two active regions compared to BPs in quiet regions (0.9 km s$^{-1}$). This is believed to be the result of a combination of an increase in magnetic flux density within the intergranular lanes in active regions confining the BPs and the disruption of normal convection leading to an increase in the number of stagnation points. The net effect then is a reduction in the width of the velocity distributions of the BPs. BPs were observed to have longer lifetimes, on average, in the two high field strength regions ($\sim$`<!-- -->`{=html}132 s) compared to the quiet region (88 s). Again, this is attributed to the higher field strength within the active region subfields, which we believe produces BPs with greater stability. As expected, a higher number density of BPs is observed in the active regions (0.38 BPs per Mm$^2$), compared to the quiet region (0.09 BPs per Mm$^2$) which is also attributed to the higher magnetic flux density within the former.

Similar to previous studies, a log-normal distribution of BP areas was observed for the active region subfields as well as for the quiet region. The peak of the distributions for the two active region subfields is both larger and broader than the quiet region subfield and suggests that BPs in active regions are $\sim$`<!-- -->`{=html}21% larger than their counterparts in quiet regions. HMI images show that the average magnetic flux density in the quiet region (average $| \Phi |$ = 7 G) was much reduced in comparison to the other two regions under investigation (average $| \Phi | \approx$ 138 G).

The diffusion index for the three regions was also calculated and was found to be about 1.2 for the three regions. This would suggest that within the three subfields, the diffusion process is slightly super-diffusive for the tracked BPs. This has been shown in previous studies on diffusivity in the photosphere. As similar values are found across the three subfields, the results support the idea that the buffeting motion of the granules controls the dynamics of the BPs. In an active region, convective processes are inhibited by the higher magnetic flux density in the region. This results in a greater number of stagnation points within the active region which acts to attenuate the velocity characteristics of BPs found in the active region, which we observe in this study. More studies should be performed to determine the extent of this effect and to what extent the higher magnetic flux density within the lanes plays a role in reducing BP velocities.

[^1]: One of the pores is outside the field-of-view of ROSA.

[^2]: Found for images degraded to the seeing conditions of their active region data set.