Authors:(financialtimes Abstract: PROOF: https://i.redd.it/deju9epm14ka1.jpgHi there, I work in the graphics department at the Financial Times website/newspaper where I have worked for over 26 years and have seen many changes in this industry over my career. The main focus of this AMA is how we have used maps during the Russia/Ukraine conflict. But you can ask me anything you want with regards to visualising data on maps, telling stories with maps, processes, software,or how I got into data visualisation.Russia’s invasion of Ukraine in maps — latest updatesKherson counter-offensiveData visualisation: how the FT newsroom designs mapsSand castles on Jersey shore: property boom defies US flood riskSuez blockage animationHow to create an animated smoke mapSmall multiple maps showing California's 22 years of dealing with droughtAnimation showing shipments of Russian fossil fuels to Europe since the invasion of UkraineAnimation showing civilian and military targets in Ukraine since the beginning of the Russian invasion3D animation of China’s nitrogen dioxide pollution levels since 2005 Read and Review the full paper at TheWinnower.com PubDate: Fri, 24 Feb 2023 10:50:25 -050
Authors:(theeconomist Abstract: Hi everyone. We're The Economist's data team. We gather, analyse and visualise data for The Economist and produce data-driven journalism. Over the past year we've created many coronavirus trackers, a risk estimator and most recently an excess-mortality model, and we've seen the interest in our work skyrocket. We can answer questions about anything relating to data journalism at The Economist. All of our work can be found on the website here or you can follow us on Twitter for updates. For more exclusive insights, sign up for our free weekly newsletter.Proof: https://twitter.com/ECONdailycharts/status/1394666569599438851's=20 Read and Review the full paper at TheWinnower.com PubDate: Thu, 20 May 2021 12:50:44 -040
Authors:me@chrisrackauckas.com (Christopher Rackauckas Abstract: Let me take a bit of time here to write out a complete canonical answer to ModelingToolkit and how it relates to Modia and Modelica. This question comes up a lot: why does ModelingToolkit exist instead of building on tooling for Modelica compilers' I'll start out by saying I am a huge fan of Martin and Hilding's work and we work very closely with them on the direction of Julia-based tooling for modeling and simulation. ModelingToolkit, being a new system, has some flexibility in the design space it explores, and while we are following a different foundational philosophy, we have many of the same goals.Composable ions for Model TransformationsEverything in the SciML organization is built around a principle of confederated modular development: let other packages influence the capabilities of your own. This is highlighted in a paper about the package structure of DifferentialEquations.jl. The underlying principle is that not everyone wants or needs to be a developer of the package, but still may want to contribute. For example, it's not uncommon that a researcher in ODE solvers wants to build a package that adds one solver to the SciML\n ecosystem. Doing this in their own package for their own academic credit, but with the free bonus that now it exists in the multiple dispatch world. In the design of DifferentialEquations.jl, solve(prob,IRKGL16()) now exists because of their package, and so we add it to the documentation. Some of this work is not even inside the organization, but we still support it. The philosophy is to include every researcher as a budding artist in the space of computational research, including all of the possible methods, and building an infrastructure that promotes a free research atmosphere in the methods. Top level defaults and documentation may lead people to the most stable aspects of the ecosystem, but with a flip of a switch you can be testing out the latest research.The Modelica ecosystem (open standard, OpenModelica, multiple commercial implementations), which started based on the simple idea of equation oriented modeling, has had a huge impact on industry and solved lots of difficult real industrial problems. The modern simulation system designer today, wants much more from their language and compiler stack. For example, in the Modelica language, there is no reference to what transformations are being done to your models in order to make them "simulatable". People know about Pantelides algorithm, and "singularity elimination", but this is outside the language. It's something that the compiler maybe gives you a few options for, but not something the user or the code actively interacts with. Every compiler is different, advances in one compiler do not help your model when you use another compiler, and the whole world is siloed. By this design, it is extremely difficult for an external user to write compiler passes in Modelica which effects this model lowering process. You can tweak knobs, or write a new compiler, or fork OpenModelica and hack on the whole… Read and Review the full paper at TheWinnower.com PubDate: Tue, 18 May 2021 07:57:16 -040
Authors:me@chrisrackauckas.com (Christopher Rackauckas Abstract: R is a widely used language for data science, but due to performance most of its underlying library are written in C, C++, or Fortran. Julia is a relative newcomer to the field which has busted out since its 1.0 to become one of the top 20 most used languages due to its high performance libraries for scientific computing and machine learning. Julia's value proposition has been its high performance in high level language, known as solving the two language problem, which has allowed allowed the language to build a robust, mature, and expansive package ecosystem. While this has been a major strength for package developers, the fact remains that there are still large and robust communities in other high level languages like R and Python. Instead of spawning distracting language wars, we should ask the question: can Julia become a language of libraries to accelerate these other languages as well'This is definitely not the first time this question was asked. The statistics libraries in Julia were developed by individuals like Douglas Bates who built some of R's most widely used packages like lme4 and Matrix. Doug had written a blog post in 2018 showing how to get top notch performance in linear mixed e\nffects model fitting via JuliaCall. In 2018 the JuliaDiffEq organization had written a blog post demonstrating the use of DifferentialEquations.jl in R and Python\n (the Jupyter of Diffrential Equations). Now rebranded as SciML for Scientific Machine Learning, we looked to expand our mission and bring automated model discovery and acceleration include other languages like R and Python with Julia as the base.With the release of diffeqr v1.0, we can now demonstrate many advances in R through the connection to Julia. Specifically, I would like to use this blog post to showcase: The new direct wrapping interface of diffeqr JIT compilation and symbolic analysis of ODEs and SDEs in R using Julia and ModelingToolkit.jl GPU-accelerated simulations of ensembles using Julia's DiffEqGPU.jlTogether we will demonstrate how models in R can be accelerated by 1000x without a user ever having to write anything but R.A Quick Note Before ContinuingBefore continuing on with showing all of the features, I wanted to ask for support so that we can continue developing these bridged libraries. Specifically, I would like to be able to support developers interested in providing a fully automated Julia installation and static compilation so that calling into Julia libraries is just as easy as any Rcpp library. To show support, the easiest thing to do is to star our libraries. The work of this blog post is build on DifferentialEquations.jl, diffeqr, ModelingToolkit.jl, and DiffEqGPU.jl. Thank you for your patience and now back to our regularly scheduled program.diffeqr v1.0: Direct wrappers for Differential Equation Solving in RFirst let me start with the new direct wrappers of differential equations solvers in R. In the previous iterations of diffeqr, we had relied on specifically designed high… Read and Review the full paper at TheWinnower.com PubDate: Tue, 18 May 2021 07:52:19 -040
Authors:me@chrisrackauckas.com (Christopher Rackauckas Abstract: Automatic differentiation is a "compiler trick" whereby a code that calculates f(x) is transformed into a code that calculates f'(x). This trick and its two forms, forward and reverse mode automatic differentiation, have become the pervasive backbone behind all of the machine learning libraries. If you ask what PyTorch or Flux.jl is doing that's special, the answer is really that it's doing automatic differentiation over some functions.What I want to dig into in this blog post is a simple question: what is the trick behind automatic differentiation, why is it always differentiation, and are there other mathematical problems we can be focusing this trick towards' While very technical discussions on this can be found in our recent paper titled "ModelingToolkit: A Composable Graph Transformation System \nFor Equation-Based Modeling" and descriptions of methods like intrusive uncertainty quantification, I want to give a high-level overview that really describes some of the intuition behind the technical thoughts. Let's dive in!What is the trick behind automatic differentiation' Non-standard interpretationTo understand automatic differentiation in practice, you need to understand that it's at its core a code transformation process. While mathematically it comes down to being about Jacobian-vector products and Jacobian-transpose-vector products for forward and reverse mode respectively, I think sometimes that mathematical treatment glosses over the practical point that it's really about code.Take for example . If we want to take the derivative of this, then we could do , but this misses the information that we actually know analytically how to define the derivative! Using the principle that algorithm efficiency comes from problem information, we can improve this process by directly embedding that analytical solution into our process. So we come to the first principle of automatic differentiation:If you know the analytical solution to the derivative, then replace the function with its derivativeSo if you see and someone calls ``derivative(f,x)``, you can do a quick little lookup to a table of rules, known as primitives, and if it's in your table then boom you're done. Swap it in, call it a day.This already shows you that, with automatic differentiation, we cannot think of as just a function, just a thing that takes in values, but we have to know something about what it means semantically. We have to look at it and identify "this is sin" in order to know "replace it with cos". This is the fundamental limitation of automatic differentiation: it has to know something about your code, more information than it takes to call or run your code. This is why many automatic differentiation libraries are tied to specific implementations of underlying numerical primitives. PyTorch understands ``torch.sin`` as , but it does not understand ``tf.sin`` as , which is why if you place a TensorFlow function into a PyTorch training loop you will get an error thrown about the derivative calculation. This semantic mapping is the reason for libraries like PubDate: Tue, 18 May 2021 07:51:46 -040
Authors:(bloomberg Abstract: For our 2020 graphic tracking real-time results, we worked with data scientist Andrew Therriault to create an exclusive voter-turnout model. Our live election map includes a state-by-state breakdown of the vote, as well as Senate, House and Governors races. We also gave readers the ability to sign up for key race alerts through a new editorial news product, Storythreads. You can read more about the methodology powering our voter turnout model here. Proof: https://twitter.com/business/status/1323770157060284424 We will be here to answer questions starting at 11:30am ET/8:30am PT Read and Review the full paper at TheWinnower.com PubDate: Thu, 05 Nov 2020 08:50:54 -050
Authors:me@chrisrackauckas.com (Christopher Rackauckas Abstract: Pre-clinical Quantitiative Systems Pharmacology (QSP) is about trying to understand how a drug target effects an outcome. If I effect this part of the biological pathways, how will it induce toxicity' Will it be effective'Recently I have been pulling in a lot of technical collegues to help with the development of next generation QSP tooling. Without a background in biological modeling, I found it difficult to explain the "how" and "why" of pharmacological modeling. Why is it differential equations, and where do these "massively expensive global optimization" runs come from' What kinds of problems can you solve with such models when you know that they are only approximate'To solve these questions, I took a step back and tried to explain a decision making scenario with a simple model, to showcase how playing with a model can allow one to distinguish between intervention strategies and uncover a way forward. This is my attempt. Instead of talking about something small and foreign like chemical reaction concentrations, let's talk about something mathematically equivalent that's easy to visualize: ecological intervention.Basic Modeling and FittingLet's take everyone's favorite ecology model: the Lotka-Volterra model. The model is the following: Left alone, the rabbit population will grow exponentially Rabbits are eaten wolves in proportion to the number of wolves (number of mouthes to feed), and in proportion to the number of rabbits (ease of food access: you eat more at a buffet!) Wolf populations grow exponentially, as long as there is a proportional amount of food around (rabbits) Wolves die overtime of old age, and any generation dies at a similar age (no major wolf medical discoveries)The model is then the ODE:using OrdinaryDiffEq, Plotsfunction f(du,u,p,t) du[1] = dx = p[1]*u[1] - p[2]*u[1]*u[2] du[2] = dy = -p[3]*u[2] + p[4]*u[1]*u[2]endu0 = [1.0;1.0]tspan = (0.0,10.0)p = [1.5,1.0,3.0,1.0]prob = ODEProblem(f,u0,tspan,p)sol = solve(prob,Tsit5())plot(sol,label=["Rabbits" "Wolves"])Except, me showing you that picture glossed over a major detail that every piece of the model is only mechanistic, but also contains a parameter. For example, rabbits grow exponentially, but what's the growth rate' To make that plot I chose a value for that growth rate (1.5), but in reality we need to get that from data since the results can be wildly different:p = [0.1,1.0,3.0,1.0]prob = ODEProblem(f,u0,tspan,p)sol = solve(prob,Tsit5())plot(sol)Here the exponential growth rate of rabbits too low to sustain a wolf population, so the wolf population dies out, but then this makes the rabbits have no predators and grow exponentially, which is a common route of ecological collapse as then they will destroy the local ecosystem. More on that later.Data and Model IssuesBut okay, we need parameters from data, but no single data source is great. One gives us a noisy sample of the population yearly, another every month for the first two years and only on the wolves, etc.:function f_true(du,u,p,t) du[1] = dx = p[1]*u[1] -… Read and Review the full paper at TheWinnower.com PubDate: Tue, 24 Mar 2020 16:07:54 -040
Authors:(ONS_UK Abstract: Hi Reddit, I’m Tom Smith, MD for the UK’s Data Science Campus as part of the Office for National Statistics. I have 20 years’ experience using data and analysis to improve public services and am a life-long data addict. I have a PhD in computational neuroscience and robotics, an MSc in knowledge-based systems and an MA in theoretical physics. I'm currently Chair of the Advisory Board to the United Nations Global Platform for big data & official statistics, Member of Council for the UK Royal Statistical Society, and previously chair of the Environment Agency Data Advisory Group, vice-chair of the Royal Statistical Society Official Statistics section, and a member of the Open Data User Group ministerial advisory group to Cabinet Office.Since the Campus was founded in 2017 we have been working on a huge range of projects including:- using tax returns, ship tracking data and road traffic sensor data to allow early identification of large economic changes;- exploring what internet traffic peaks and troughs can tell us about our lives; - using satellite imagery to detect surface water and assess changes over time, for rapid detection of emerging issues;- launching a hub focused on data science and AI for International Development, located at the Department for International Development (DfID), near Glasgow.- supporting ONS, government and public sector organisations to increase their data science capability. We’re aiming to have 500 trained data science practitioners for UK government by 2021.I'll be here to talk about statistics, data and making the world a better place from 3-5pm GMT today. Proof: https://twitter.com/ONSfocus/status/1237060713140625416Ask me anything! Read and Review the full paper at TheWinnower.com PubDate: Mon, 16 Mar 2020 06:50:40 -040
Open Access journal
. If we want to take the derivative of this, then we could do 
, but this misses the information that we actually know analytically how to define the derivative! Using the principle that algorithm efficiency comes from problem information, we can improve this process by directly embedding that analytical solution into our process. So we come to the first principle of automatic differentiation:If you know the analytical solution to the derivative, then replace the function with its derivativeSo if you see 
as just a function, just a thing that takes in values, but we have to know something about what it means semantically. We have to look at it and identify "this is sin" in order to know "replace it with cos". This is the fundamental limitation of automatic differentiation: it has to know something about your code, more information than it takes to call or run your code. This is why many automatic differentiation libraries are tied to specific implementations of underlying numerical primitives. PyTorch understands ``torch.sin`` as 
, but it does not understand ``tf.sin`` as 
