Toward optimally distributed computation.

P J Edwards; A F Murray

doi:10.1162/089976698300017593

Home/Archive/PubMed

PubMed · 9573416

Toward optimally distributed computation.

P J Edwards·A F Murray

View original source ↗Export metadata ↓

Abstract

This article introduces the concept of optimally distributed computation in feedforward neural networks via regularization of weight saliency. By constraining the relative importance of the parameters, computation can be distributed thinly and evenly throughout the network. We propose that this will have beneficial effects on fault-tolerance performance and generalization ability in large network architectures. These theoretical predictions are verified by simulation experiments on two problems: one artificial and the other a real-world task. In summary, this article presents regularization terms for distributing neural computation optimally.

Explore related subjects

Computer Simulation Models, Statistical Neural Networks, Computer Reproducibility of Results Journal Article Research Support, Non-U.S. Gov't

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

P J Edwards, A F Murray. 1998-05-15. Toward optimally distributed computation.. https://doi.org/10.1162/089976698300017593

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2025

Generating correlated data for omics simulation.

Simulation of realistic omics data is a key input for benchmarking studies that help users obtain optimal computational pipelines. Omics data involves large numbers of measured features on each sample and these measures are generally correlated with each other. However, simulation too often ignores these correlations, perhaps due to computational and statistical hurdles of doing so. To alleviate this, we describe three approaches for generating omics-scale data with correlated measures which mimic real datasets. These approaches are all based on a Gaussian copula approach with a covariance matrix that decomposes into a diagonal part and a low-rank part. This decomposition allows for extremely efficient simulation, overcoming a hurdle for adoption of past methods. We use these approaches to demonstrate the importance of including correlation in two benchmarking applications. First, we show that variance of results from the popular DESeq2 method increases when dependence is included. Second, we demonstrate that CYCLOPS, a method for inferring circadian time of collection from transcriptomics, improves in performance when given gene-gene dependencies in some circumstances. We provide an R package, dependentsimr, that has efficient implementations of these methods and can generate dependent data with arbitrary marginal distributions, including discrete (binary, ordered categorical, Poisson, negative binomial), continuous (normal), or with an empirical distribution.

Computer Simulation↗

PubMed2017

Addressing current challenges in cancer immunotherapy with mathematical and computational modelling.

The goal of cancer immunotherapy is to boost a patient's immune response to a tumour. Yet, the design of an effective immunotherapy is complicated by various factors, including a potentially immunosuppressive tumour microenvironment, immune-modulating effects of conventional treatments and therapy-related toxicities. These complexities can be incorporated into mathematical and computational models of cancer immunotherapy that can then be used to aid in rational therapy design. In this review, we survey modelling approaches under the umbrella of the major challenges facing immunotherapy development, which encompass tumour classification, optimal treatment scheduling and combination therapy design. Although overlapping, each challenge has presented unique opportunities for modellers to make contributions using analytical and numerical analysis of model outcomes, as well as optimization algorithms. We discuss several examples of models that have grown in complexity as more biological information has become available, showcasing how model development is a dynamic process interlinked with the rapid advances in tumour-immune biology. We conclude the review with recommendations for modellers both with respect to methodology and biological direction that might help keep modellers at the forefront of cancer immunotherapy development.

Computer Simulation↗

PubMed2000

Future vision of the GDB human genome database.

In 1973, scientists assembled at the first Human Gene Mapping Workshop to discuss the 64 human genes mapped at that time. In 1989, the GDB Human Genome Database was created to store information on 1, 700 mapped human genes. Ten years later, as the human genome project closes in on the release of the complete DNA sequence holding as many as 100,000 human genes, GDB is evolving to continue to meet the needs of the scientific community. Well known as a resource for data which has been stringently reviewed as part of the curation process, GDB prepares to continue to provide a compilation of the human genome including maps, map objects, polymorphisms, and mutations. As more sites across the Internet are established to share biological information, it becomes increasingly burdensome for the scientist to collect data from all sources of a particular domain. In an attempt to reduce this burden, GDB continues to load data from large genome centres and accept submissions from researchers around the world. Moreover, GDB looks to provide a mechanism to link gene-related information to the human reference sequence. In doing this, GDB plans to establish federated linkages with "boutique" databases around the world that could contain enormous amounts of valuable information about specific genes or chromosomes.

Computer Simulation↗