Keep the gradient flowinghttp://fa.bianp.net/Sat, 04 May 2024 00:00:00 +0200On the Link Between Optimization and Polynomials, Part 6.http://fa.bianp.net/blog/2024/unrolling/ <p> Differentiating through optimization is a fundamental problem in hyperparameter optimization, dataset distillation, meta-learning and optimization as a layer, to name a few. In this blog post we'll look into one of the main approaches to differentiate through optimization: unrolled differentiation. With the help of polynomials, we'll be able to derive …</p>Fabian PedregosaSat, 04 May 2024 00:00:00 +0200tag:fa.bianp.net,2024-05-04:/blog/2024/unrolling/optimizationpolynomialsbi-levelOptimization Nuggets: Stochastic Polyak Step-size, Part 2http://fa.bianp.net/blog/2023/sps2/ <p> This blog post discusses the convergence rate of the Stochastic Gradient Descent with Stochastic Polyak Step-size (SGD-SPS) algorithm for minimizing a finite sum objective. Building upon the proof of the previous post, we show that the convergence rate can be improved to O(1/t) under the additional assumption that …</p>Fabian Pedregosa and <a href='https://fabian-sp.github.io/'>Fabian Schaipp</a>Sun, 19 Nov 2023 00:00:00 +0100tag:fa.bianp.net,2023-11-19:/blog/2023/sps2/optimizationSGDproofsOptimization Nuggets: Stochastic Polyak Step-sizehttp://fa.bianp.net/blog/2023/sps/ <p> The stochastic Polyak step-size (SPS) is a practical variant of the Polyak step-size for stochastic optimization. In this blog post, we'll discuss the algorithm and provide a simple analysis for convex objectives with bounded gradients. </p> <script type="text/javascript" src="/theme/js/bibtexParse.js"> </script> <script> MathJax = { tex: { inlineMath: [['$', '$'], ['\$$', '\$$']], tags: 'ams' }, svg: { fontCache: 'global' } }; </script> <script type="text/javascript" id="MathJax-script" async src="/node_modules/mathjax3/es5/tex-svg.js"> </script> <script type="text/javascript" src="/theme/js/refs_v1.js"></script> <script type="text/javascript"> const bibtex =  @inproceedings{gunasekar2018characterizing, title={Characterizing implicit …</script>Fabian PedregosaFri, 29 Sep 2023 00:00:00 +0200tag:fa.bianp.net,2023-09-29:/blog/2023/sps/optimizationSGDproofsOn the Convergence of the Unadjusted Langevin Algorithmhttp://fa.bianp.net/blog/2023/ulaq/ <p> The Langevin algorithm is a simple and powerful method to sample from a probability distribution. It's a key ingredient of some machine learning methods such as diffusion models and differentially private learning. In this post, I'll derive a simple convergence analysis of this method in the special case when the …</p><a href='http://fa.bianp.net/pages/about.html'>Fabian Pedregosa</a>Wed, 14 Jun 2023 00:00:00 +0200tag:fa.bianp.net,2023-06-14:/blog/2023/ulaq/samplingLangevindiffusionThe Russian Roulette: An Unbiased Estimator of the Limithttp://fa.bianp.net/blog/2022/russian-roulette/ <blockquote class="pullquote" style="margin-left: 20px"> <p> <q>The idea for what was later called Monte Carlo method occurred to me when I was playing solitaire during my illness.</q> </p> <p style="text-align: right;"> Stanislaw Ulam, <i><a href="https://www.goodreads.com/book/show/423246.Adventures_of_a_Mathematician">Adventures of a Mathematician</a></i> </p> </blockquote> <p> The Russian Roulette offers a simple way to construct an unbiased estimator for the limit of a sequence. It allows for example to …</p>Fabian PedregosaSat, 15 Oct 2022 00:00:00 +0200tag:fa.bianp.net,2022-10-15:/blog/2022/russian-roulette/statisticsnuggetsunbiasedNotes on the Frank-Wolfe Algorithm, Part III: backtracking line-searchhttp://fa.bianp.net/blog/2022/adaptive_fw/ <p> Backtracking step-size strategies (also known as adaptive step-size or approximate line-search) that set the step-size based on a sufficient decrease condition are the standard way to set the step-size on gradient descent and quasi-Newton methods. However, these techniques are much less common for <a href="/blog/2018/notes-on-the-frank-wolfe-algorithm-part-i/">Frank-Wolfe-like</a> algorithms. In this blog post I …</p>Fabian PedregosaFri, 26 Aug 2022 00:00:00 +0200tag:fa.bianp.net,2022-08-26:/blog/2022/adaptive_fw/optimizationFrank-Wolfebacktrackingline-searchstep-sizeadaptiveOn the Link Between Optimization and Polynomials, Part 5http://fa.bianp.net/blog/2022/cyclical/ <blockquote class="pullquote" style="margin-left: 20px"> <p> <br><i> <b style="font-style: normal;">Six</b>: All of this has happened before. <br> <b style="font-style: normal;">Baltar</b>: But the question remains, does all of this have to happen again?<br> <b style="font-style: normal;">Six</b>: This time I bet no.<br> <b style="font-style: normal;">Baltar</b>: You know, I've never known you to play the optimist. Why the change of heart?<br> <b style="font-style: normal;">Six</b>: Mathematics. Law of averages. Let a complex …</i></p></blockquote><a href='https://scholar.google.com/citations?user=93PAG2AAAAAJ&hl=en'>Baptiste Goujaud</a> and <a href='http://fa.bianp.net/pages/about.html'>Fabian Pedregosa</a>Fri, 27 May 2022 00:00:00 +0200tag:fa.bianp.net,2022-05-27:/blog/2022/cyclical/optimizationpolynomialsaccelerationOptimization Nuggets: Implicit Bias of Gradient-based Methodshttp://fa.bianp.net/blog/2022/implicit-bias-regression/ <p> When an optimization problem has multiple global minima, different algorithms can find different solutions, a phenomenon often referred to as the <i>implicit bias</i> of optimization algorithms. In this post we'll characterize the implicit bias of gradient-based methods on a class of regression problems that includes linear least squares and Huber …</p>Fabian PedregosaMon, 10 Jan 2022 00:00:00 +0100tag:fa.bianp.net,2022-01-10:/blog/2022/implicit-bias-regression/learning theoryimplicit biasproofsnuggetsOptimization Nuggets: Exponential Convergence of SGDhttp://fa.bianp.net/blog/2021/exponential-sgd/ <p> This is the first of a series of blog posts on short and beautiful proofs in optimization (let me know what you think in the comments!). For this first post in the series I'll show that stochastic gradient descent (SGD) converges exponentially fast to a neighborhood of the solution. </p> <script type="text/javascript" src="/theme/js/bibtexParse.js"> </script> <script type="text/x-mathjax-config"> MathJax …</script>Fabian PedregosaWed, 15 Dec 2021 00:00:00 +0100tag:fa.bianp.net,2021-12-15:/blog/2021/exponential-sgd/optimizationSGDproofsnuggetsOn the Link Between Optimization and Polynomials, Part 4http://fa.bianp.net/blog/2021/no-momentum/ <p> While the most common accelerated methods like Polyak and Nesterov incorporate a momentum term, a little known fact is that simple gradient descent &ndash;no momentum&ndash; can achieve the same rate through only a well-chosen sequence of step-sizes. In this post we'll derive this method and through simulations discuss its practical …</p>Fabian PedregosaTue, 13 Apr 2021 00:00:00 +0200tag:fa.bianp.net,2021-04-13:/blog/2021/no-momentum/optimizationpolynomialsaccelerationOn the Link Between Optimization and Polynomials, Part 3http://fa.bianp.net/blog/2021/hitchhiker/ <blockquote class="pullquote"> <p> <q><i>I've seen things you people wouldn't believe. <br> Valleys sculpted by trigonometric functions. <br> Rates on fire off the shoulder of divergence. <br> Beams glitter in the dark near the Polyak gate. <br> All those landscapes will be lost in time, like tears in rain. <br>Time to halt.</i></q> <br> </p> <p style="text-align: right;"> A momentum optimizer <a href="https://en.wikipedia.org/wiki/Tears_in_rain_monologue">*</a> </p> </blockquote> <figure class="fullwidth"> <img style="max-width: 1000px;" src="/images/2021/rate_convergence_momentum.png" alt=""> </figure> <script type="text/javascript" src="/theme/js/bibtexParse.js"> </script> <script type="text/x-mathjax-config"> MathJax.Hub.Config …</script>Fabian PedregosaTue, 02 Mar 2021 00:00:00 +0100tag:fa.bianp.net,2021-03-02:/blog/2021/hitchhiker/optimizationpolynomialsaccelerationmomentumChebyshevOn the Link Between Optimization and Polynomials, Part 2http://fa.bianp.net/blog/2020/momentum/ <p> We can tighten the analysis of gradient descent with momentum through a cobination of Chebyshev polynomials of the first and second kind. Following this connection, we'll derive one of the most iconic methods in optimization: Polyak momentum. </p> <script type="text/javascript" src="/theme/js/bibtexParse.js"> </script> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath …</script>Fabian PedregosaMon, 21 Dec 2020 00:00:00 +0100tag:fa.bianp.net,2020-12-21:/blog/2020/momentum/optimizationpolynomialsaccelerationmomentumChebyshevOn the Link Between Polynomials and Optimization, Part 1http://fa.bianp.net/blog/2020/polyopt/ <p> There's a fascinating link between minimization of quadratic functions and polynomials. A link that goes deep and allows to phrase optimization problems in the language of polynomials and vice versa. Using this connection, we can tap into centuries of research in the theory of polynomials and shed new light on …</p>Fabian PedregosaTue, 07 Apr 2020 00:00:00 +0200tag:fa.bianp.net,2020-04-07:/blog/2020/polyopt/optimizationpolynomialsaccelerationChebyshevHow to Evaluate the Logistic Loss and not NaN tryinghttp://fa.bianp.net/blog/2019/evaluate_logistic/ <p>A naive implementation of the logistic regression loss can results in numerical indeterminacy even for moderate values. This post takes a closer look into the source of these instabilities and discusses more robust Python implementations. </p> <!-- for highlighting --> <link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.5/styles/default.min.css"> <script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.15.5/highlight.min.js"></script> <script>hljs.initHighlightingOnLoad();</script> <!-- Mathjax--> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath …</script><a href='http://fa.bianp.net/pages/about.html'>Fabian Pedregosa</a>Fri, 27 Sep 2019 00:00:00 +0200tag:fa.bianp.net,2019-09-27:/blog/2019/evaluate_logistic/codinglogistic regressionnumerical stabilityNotes on the Frank-Wolfe Algorithm, Part II: A Primal-dual Analysishttp://fa.bianp.net/blog/2018/fw2/ <p>This blog post extends the convergence theory from the <a href="/blog/2018/notes-on-the-frank-wolfe-algorithm-part-i/">first part of these notes</a> on the Frank-Wolfe (FW) algorithm with convergence guarantees on the primal-dual gap which generalize and strengthen the convergence guarantees obtained in the first part. </p> <script type="text/javascript" src="/theme/js/bibtexParse.js"> </script> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax …</script>Fabian PedregosaSat, 17 Nov 2018 00:00:00 +0100tag:fa.bianp.net,2018-11-17:/blog/2018/fw2/optimizationFrank-Wolfeconditional gradientconvergence analysisThree Operator Splittinghttp://fa.bianp.net/blog/2018/tos/I discuss a recently proposed optimization algorithm: the Davis-Yin three operator splitting.Fabian PedregosaThu, 06 Sep 2018 00:00:00 +0200tag:fa.bianp.net,2018-09-06:/blog/2018/tos/optimizationproximal splittingthree operator splittingconvergence analysisNotes on the Frank-Wolfe Algorithm, Part Ihttp://fa.bianp.net/blog/2018/notes-on-the-frank-wolfe-algorithm-part-i/ <p>This blog post is the first in a series discussing different theoretical and practical aspects of the Frank-Wolfe algorithm.</p> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ tex2jax: { inlineMath: [ ['$','$'], ["\$$","\$$"] ], displayMath: [ ['$$','$$'], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js?config=TeX-AMS-MML_HTMLorMML"> </script> <!-- for highlighting --> <link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/default.min.css"> <script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script> <script>hljs.initHighlightingOnLoad();</script> <div style="display: none"> $$\def\xx{\boldsymbol x} \def\yy{\boldsymbol y} \def\ss{\boldsymbol s} \def\dd …</div>Fabian PedregosaWed, 21 Mar 2018 00:00:00 +0100tag:fa.bianp.net,2018-03-21:/blog/2018/notes-on-the-frank-wolfe-algorithm-part-i/optimizationFrank-Wolfeconditional gradientconvergence analysisOptimization inequalities cheatsheethttp://fa.bianp.net/blog/2017/optimization-inequalities-cheatsheet/ <p>Most proofs in optimization consist in using inequalities for a particular function class in some creative way. This is a cheatsheet with inequalities that I use most often. It considers class of functions that are convex, strongly convex and L-smooth. </p> <script type="text/javascript" src="/theme/js/bibtexParse.js"> </script> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX …</script>Fabian PedregosaWed, 11 Jan 2017 00:00:00 +0100tag:fa.bianp.net,2017-01-11:/blog/2017/optimization-inequalities-cheatsheet/optimizationoptimizationcheatsheetA fully asynchronous variant of the SAGA algorithmhttp://fa.bianp.net/blog/2016/a-fully-asynchronous-variant-of-the-saga-algorithm/<p>My friend <a href="http://www.di.ens.fr/~rleblond/">Rémi Leblond</a> has recently uploaded to ArXiv <a href="https://arxiv.org/abs/1606.04809">our preprint on an asynchronous version of the SAGA optimization algorithm</a>.</p> <p>The main contribution is to develop a parallel (fully asynchronous, no locks) variant of the <a href="http://papers.nips.cc/paper/5258-saga-a-fast-incremental-gradient-method-with-support-for-non-strongly-convex-composite-objectives.pdf">SAGA algorighm</a>. This is a stochastic variance-reduced method for general optimization, specially adapted for problems …</p>Fabian PedregosaWed, 12 Oct 2016 00:00:00 +0200tag:fa.bianp.net,2016-10-12:/blog/2016/a-fully-asynchronous-variant-of-the-saga-algorithm/optimizationoptimizationasynchronousSAGAHyperparameter optimization with approximate gradienthttp://fa.bianp.net/blog/2016/hyperparameter-optimization-with-approximate-gradient/ <script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''], ["\$$","\$$"] ], displayMath: [ ['$$','$$'], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, Macros: { RR: "{\\mathbb{R}}", argmin: "{\\mathop{\\mathrm{arg\\,min}}}", bold: ["{\\bf #1}",1] } }, "HTML-CSS": { availableFonts: ["TeX"] } }); </script> <script type="text/javascript" async src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> </script> <p><b>TL;DR:</b> I describe a <a href="http://arxiv.org/abs/1602.02355">method for hyperparameter optimization</a> by gradient descent.</p> <p>Most machine …</p>Fabian PedregosaWed, 25 May 2016 00:00:00 +0200tag:fa.bianp.net,2016-05-25:/blog/2016/hyperparameter-optimization-with-approximate-gradient/optimizationmachine learninghyperparametersHOAGLightning v0.1http://fa.bianp.net/blog/2016/lightning-v01/<p>Announce: first public release of <a href="http://contrib.scikit-learn.org/lightning/">lightning</a>!, a library for large-scale linear classification, regression and ranking in Python. The library was started a couple of years ago by <a href="http://mblondel.org">Mathieu Blondel</a> who also contributed the vast majority of source code. I joined recently its development and decided it was about time for …</p>Fabian PedregosaFri, 25 Mar 2016 00:00:00 +0100tag:fa.bianp.net,2016-03-25:/blog/2016/lightning-v01/softwarePythonscikit-learnmachine learninglightningscikit-learn-contrib, an umbrella for scikit-learn related projects.http://fa.bianp.net/blog/2016/scikit-learn-contrib-an-umbrella-for-scikit-learn-related-projects/<p>Together with other scikit-learn developers we've created an umbrella organization for scikit-learn-related projects named <a href="https://github.com/scikit-learn-contrib">scikit-learn-contrib</a>. The idea is for this organization to host projects that are deemed too specific or too experimental to be included in the scikit-learn codebase but still offer an API which is compatible with scikit-learn and …</p>Fabian PedregosaSun, 06 Mar 2016 00:00:00 +0100tag:fa.bianp.net,2016-03-06:/blog/2016/scikit-learn-contrib-an-umbrella-for-scikit-learn-related-projects/softwarePythonscikit-learnmachine learninglightningSAGA algorithm in the lightning libraryhttp://fa.bianp.net/blog/2016/saga-algorithm-in-the-lightning-library/<p>Recently I've implemented, together with <a href="http://arachez.com/">Arnaud Rachez</a>, the SAGA[<sup id="fnref:1"><a class="footnote-ref" href="#fn:1">1</a></sup>] algorithm in the <a href="http://contrib.scikit-learn.org/lightning/">lightning</a> machine learning library (which by the way, has been recently moved to the new <a href="https://github.com/scikit-learn-contrib">scikit-learn-contrib</a> project). The lightning library uses the same API as scikit-learn but is particularly adapted to online learning. As for the SAGA …</p>Fabian PedregosaMon, 22 Feb 2016 00:00:00 +0100tag:fa.bianp.net,2016-02-22:/blog/2016/saga-algorithm-in-the-lightning-library/miscPythonscikit-learnmachine learninglightningOn the consistency of ordinal regression methodshttp://fa.bianp.net/blog/2015/on-the-consistency-of-ordinal-regression-methods/<script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''], ["\$$","\$$"] ], displayMath: [ ['$$',''], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, extensions: ["AMSmath.js", "AMSsymbols.js"] }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js"> </script> <p>My latests work (with <a href="http://www.di.ens.fr/~fbach/">Francis Bach</a> and <a href="http://alexandre.gramfort.net/">Alexandre Gramfort</a>) is on the consistency of ordinal regression methods. It has the wildly imaginative …</p>Fabian PedregosaFri, 09 Oct 2015 00:00:00 +0200tag:fa.bianp.net,2015-10-09:/blog/2015/on-the-consistency-of-ordinal-regression-methods/learning theoryconsistencymachine learningHoldout cross-validation generatorhttp://fa.bianp.net/blog/2015/holdout-cross-validation-generator/<p><a href="http://scikit-learn.org/stable/modules/cross_validation.html#cross-validation-iterators">Cross-validation iterators</a> in scikit-learn are simply generator objects, that is, Python objects that implement the <code>__iter__</code> method and that for each call to this method return (or more precisely, <code>yield</code>) the indices or a boolean mask for the train and test set. Hence, implementing new cross-validation iterators that behave as …</p>Fabian PedregosaThu, 20 Aug 2015 00:00:00 +0200tag:fa.bianp.net,2015-08-20:/blog/2015/holdout-cross-validation-generator/miscPythonscikit-learnmachine learningmodel selectionIPython/Jupyter notebook galleryhttp://fa.bianp.net/blog/2015/ipythonjupyter-notebook-gallery/<p style="font-weight: bold; color:red">Due to lack of time and interest, I'm no longer maintaining this project. Feel free to grab the sources from <a href="https://github.com/fabianp/nbgallery">https://github.com/fabianp/nbgallery</a> and fork the project. </p> <p>TL;DR I created a gallery for IPython/Jupyter notebooks. <a href="http://nb.bianp.net">Check it out :-)</a></p> <div style="text-align: center"> <img alt="Notebook gallery" width="600px" src="http://fa.bianp.net/uploads/2015/screenshot_nbgallery.png" /> </div> <p>A couple of months ago I put online …</p>Fabian PedregosaTue, 21 Apr 2015 00:00:00 +0200tag:fa.bianp.net,2015-04-21:/blog/2015/ipythonjupyter-notebook-gallery/miscPythonJupyterPyData Paris - April 2015http://fa.bianp.net/blog/2015/pydata-paris-april-2015/<p>Last Friday was <a href="http://pydataparis.joinux.org/schedule.html">PyData Paris</a>, in words of the organizers, ''a gathering of users and developers of data analysis tools in Python''. </p> <p><img width="600px" src="http://pydataparis.joinux.org/static/images/PyDataLogoBig-Paris2015.png" /></p> <p>The organizers did a great job in putting together and the event started already with a full room for <a href="http://gael-varoquaux.info/">Gael's</a> keynote</p> <div style="text-align: center"> <img width="600px" alt="Gael's keynote" src="https://pbs.twimg.com/media/CBplCb_WIAEzytd.jpg" /></div> <p>My take-away message from the talks is …</p>Fabian PedregosaTue, 07 Apr 2015 00:00:00 +0200tag:fa.bianp.net,2015-04-07:/blog/2015/pydata-paris-april-2015/miscPythonParisNumPyNumbaData-driven hemodynamic response function estimationhttp://fa.bianp.net/blog/2014/data-driven-hemodynamic-response-function-estimation/<p>My <a href="http://www.sciencedirect.com/science/article/pii/S1053811914008027">latest research paper</a>[<sup id="fnref:1"><a class="footnote-ref" href="#fn:1">1</a></sup>] deals with the estimation of the hemodynamic response function (HRF) from fMRI data. </p> <div style="text-align: center"> <img width="600px" src="/images/2014/graphical_abstract.jpg" /> </div> <p>This is an important topic since the knowledge of a hemodynamic response function is what makes it possible to extract the brain activation maps that are used in most of the impressive …</p>Fabian PedregosaFri, 05 Dec 2014 00:00:00 +0100tag:fa.bianp.net,2014-12-05:/blog/2014/data-driven-hemodynamic-response-function-estimation/miscfMRIGLMpythonPlot memory usage as a function of timehttp://fa.bianp.net/blog/2014/plot-memory-usage-as-a-function-of-time/<p>:og_image: http://fa.bianp.net/blog/images/2014/mprof_example.png</p> <p>One of the lesser known features of the <a href="https://pypi.python.org/pypi/memory_profiler">memory_profiler package</a> is its ability to plot memory consumption as a function of time. This was implemented by my friend Philippe Gervais, previously a colleague at INRIA and now at Google.</p> <p>With …</p>Fabian PedregosaFri, 07 Nov 2014 00:00:00 +0100tag:fa.bianp.net,2014-11-07:/blog/2014/plot-memory-usage-as-a-function-of-time/miscmemory_profilermprofprofileSurrogate Loss Functions in Machine Learninghttp://fa.bianp.net/blog/2014/surrogate-loss-functions-in-machine-learning/<!-- <div style="float: left; margin: 20px; width; 200px" > <img src="http://upload.wikimedia.org/wikipedia/commons/4/46/R._A._Fischer.jpg" /> <p>Sir R. A. Fisher. Source: Wikipedia </p> </div> --> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''], ["\$$","\$$"] ], displayMath: [ ['','$$'], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, extensions: ["AMSmath.js", "AMSsymbols.js"] }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js"> </script> <p><span class="bold">TL; DR</span> These are some notes on calibration of surrogate loss functions in the context of machine learning. But mostly it is …</p>Fabian PedregosaFri, 20 Jun 2014 00:00:00 +0200tag:fa.bianp.net,2014-06-20:/blog/2014/surrogate-loss-functions-in-machine-learning/miscmachine learningconsistencycalibrationDifferent ways to get memory consumption or lessons learned from memory_profiler`http://fa.bianp.net/blog/2013/different-ways-to-get-memory-consumption-or-lessons-learned-from-memory_profiler/<p>As part of the development of <a href="https://pypi.python.org/pypi/memory_profiler">memory_profiler</a> I've tried several ways to get memory usage of a program from within Python. In this post I'll describe the different alternatives I've tested.</p> <h3>The psutil library</h3> <p><a href="https://code.google.com/p/psutil/">psutil</a> is a python library that provides an interface for retrieving information on running processes. It …</p>Fabian PedregosaThu, 25 Jul 2013 00:00:00 +0200tag:fa.bianp.net,2013-07-25:/blog/2013/different-ways-to-get-memory-consumption-or-lessons-learned-from-memory_profiler/miscPythonmemorymemory_profilerNumerical optimizers for Logistic Regressionhttp://fa.bianp.net/blog/2013/numerical-optimizers-for-logistic-regression/<script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''], ["\$$","\$$"] ], displayMath: [ ['$$','$$'], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, extensions: ["AMSmath.js", "AMSsymbols.js"] }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js"> </script> <p>In this post I compar several implementations of Logistic Regression. The task was to implement a Logistic Regression model using standard optimization …</p>Fabian PedregosaMon, 20 May 2013 00:00:00 +0200tag:fa.bianp.net,2013-05-20:/blog/2013/numerical-optimizers-for-logistic-regression/miscmachine learninglogistic regressionPythonSciPyLogistic Ordinal Regressionhttp://fa.bianp.net/blog/2013/logistic-ordinal-regression/<p><strong>TL;DR: I've implemented a logistic ordinal regression or proportional odds model. <a href="http://github.com/fabianp/minirank/blob/master/minirank/logistic.py">Here is the Python code</a></strong></p> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''] ], displayMath: [ ['$$','$$'], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, extensions: ["AMSmath.js", "AMSsymbols.js"] }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js"> </script> <p>The <em>logistic ordinal regression</em> model …</p>Fabian PedregosaThu, 02 May 2013 00:00:00 +0200tag:fa.bianp.net,2013-05-02:/blog/2013/logistic-ordinal-regression/miscmachine learningordinal regressionPythonrankingIsotonic Regressionhttp://fa.bianp.net/blog/2013/isotonic-regression/<script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''], ["\$$","\$$"] ], displayMath: [ ['$$','$$'], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, extensions: ["AMSmath.js", "AMSsymbols.js"] }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js"> </script> <p>My latest contribution for <a href="http://scikit-learn.org">scikit-learn</a> is an implementation of the isotonic regression model that I coded with <a href="https://twitter.com/nvaroqua">Nelle Varoquaux</a> and <a href="http://alexandre.gramfort.net/">Alexandre Gramfort …</a></p>Fabian PedregosaTue, 16 Apr 2013 00:00:00 +0200tag:fa.bianp.net,2013-04-16:/blog/2013/isotonic-regression/miscisotonic regressionmachine learningPythonscikit-learnHouseholder matriceshttp://fa.bianp.net/blog/2013/householder-matrices/<script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''], ["\$$","\$$"] ], displayMath: [ ['$$','$$'], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, extensions: ["AMSmath.js", "AMSsymbols.js"] }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js"> </script> <p>Householder matrices are square matrices of the form</p> <p>$$ P = I - \beta v v^T$$</p> <p>where \beta is a scalar and v is …</p>Fabian PedregosaSat, 30 Mar 2013 00:00:00 +0100tag:fa.bianp.net,2013-03-30:/blog/2013/householder-matrices/misclinear algebrahouseholderQRLoss Functions for Ordinal regressionhttp://fa.bianp.net/blog/2013/loss-functions-for-ordinal-regression/<script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''], ["\$$","\$$"] ], displayMath: [ ['$$','$$'], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, extensions: ["AMSmath.js", "AMSsymbols.js"] }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js"> </script> <p>** Note: this post contains a fair amount of LaTeX, if you don't visualize the math correctly come to its <a href="http://fa.bianp.net/blog/2013/loss-functions-for-ordinal-regression/">original location</a> **</p> <p>In …</p>Fabian PedregosaWed, 27 Feb 2013 00:00:00 +0100tag:fa.bianp.net,2013-02-27:/blog/2013/loss-functions-for-ordinal-regression/miscmachine learningordinal regressionloss functionMemory plots with memory_profilerhttp://fa.bianp.net/blog/2013/memory-plots-with-memory_profiler/<p>Besides performing a line-by-line analysis of memory consumption, <a href="http://pypi.python.org/pypi/memory_profiler"><code>memory_profiler</code></a> exposes some functions that allow to retrieve the memory consumption of a function in real-time, allowing e.g. to visualize the memory consumption of a given function over time.</p> <p>The function to be used is <code>memory_usage</code>. The first argument specifies what …</p>Fabian PedregosaFri, 04 Jan 2013 00:00:00 +0100tag:fa.bianp.net,2013-01-04:/blog/2013/memory-plots-with-memory_profiler/miscPythonmemorymemory_profilerSingular Value Decomposition in SciPyhttp://fa.bianp.net/blog/2012/singular-value-decomposition-in-scipy/<p>SciPy contains two methods to compute the singular value decomposition (SVD) of a matrix: <code>scipy.linalg.svd</code> and <code>scipy.sparse.linalg.svds</code>. In this post I'll compare both methods for the task of computing the full SVD of a large dense matrix.</p> <p>The first method, <code>scipy.linalg.svd</code>, is perhaps …</p>Fabian PedregosaSat, 08 Dec 2012 00:00:00 +0100tag:fa.bianp.net,2012-12-08:/blog/2012/singular-value-decomposition-in-scipy/miscpythonscipysvdLearning to rank with scikit-learn: the pairwise transformhttp://fa.bianp.net/blog/2012/learning-to-rank-with-scikit-learn-the-pairwise-transform/<script type="text/x-mathjax-config"> MathJax.Hub.Config({ extensions: ["tex2jax.js"], jax: ["input/TeX", "output/HTML-CSS"], tex2jax: { inlineMath: [ ['',''], ["\$$","\$$"] ], displayMath: [ ['$$',''], ["\$","\$"] ], processEscapes: true }, TeX: { equationNumbers: { autoNumber: "AMS" }, extensions: ["AMSmath.js", "AMSsymbols.js"] }, "HTML-CSS": { fonts: ["TeX"] } }); </script> <script type="text/javascript" async src="/node_modules/mathjax2/MathJax.js"> </script> <p>This tutorial introduces the concept of pairwise preference used in most <a href="http://en.wikipedia.org/wiki/Learning_to_rank">ranking problems</a>. I'll use scikit-learn and for learning and matplotlib for …</p>Fabian PedregosaTue, 23 Oct 2012 00:00:00 +0200tag:fa.bianp.net,2012-10-23:/blog/2012/learning-to-rank-with-scikit-learn-the-pairwise-transform/miscpythonscikit-learnrankingline-by-line memory usage of a Python programhttp://fa.bianp.net/blog/2012/line-by-line-report-of-memory-usage/<p>My newest project is a Python library for monitoring memory consumption of arbitrary process, and one of its most useful features is the line-by-line analysis of memory usage for Python code. I wrote a basic prototype six months ago after being surprised by the lack of related tools. I wanted …</p>Fabian PedregosaTue, 24 Apr 2012 07:04:00 +0200tag:fa.bianp.net,2012-04-24:/blog/2012/line-by-line-report-of-memory-usage/miscpythonmemory_profilerLow rank approximationhttp://fa.bianp.net/blog/2011/low-rank-approximation/<p>A little experiment to see what low rank approximation looks like. These are the best rank-k approximations (in the Frobenius norm) to the a natural image for increasing values of k and an original image of rank 512.</p> <img alt="" src="/blog/static/uploads/2011/11/animation1.gif" /> <p>Python code can be found <a class="reference external" href="https://gist.github.com/1342033">here</a>. GIF animation made using ImageMagic's convert …</p>Fabian PedregosaSun, 06 Nov 2011 12:05:00 +0100tag:fa.bianp.net,2011-11-06:/blog/2011/low-rank-approximation/miscmachine learningpythonqr_multiply function in scipy.linalghttp://fa.bianp.net/blog/2011/qr_multiply-function-in-scipylinalg/<p>In scipy's development version there's a new function closely related to the <a class="reference external" href="http://en.wikipedia.org/wiki/QR_decomposition">QR-decomposition</a> of a matrix and to the least-squares solution of a linear system. What this function does is to compute the QR-decomposition of a matrix and then multiply the resulting orthogonal factor by another arbitrary matrix. In pseudocode …</p>Fabian PedregosaFri, 14 Oct 2011 16:44:00 +0200tag:fa.bianp.net,2011-10-14:/blog/2011/qr_multiply-function-in-scipylinalg/miscpythonscipyscikit-learn 0.9http://fa.bianp.net/blog/2011/scikit-learn-09/<p>Last week we released a new version of scikit-learn. The <a class="reference external" href="http://scikit-learn.sourceforge.net/stable/whats_new.html">Changelog is particularly impressive</a>, yet personally this release is important for other reasons. This will probably be my last release as a paid engineer. I'm starting a PhD next month, and although I plan to continue contributing to the project …</p>Fabian PedregosaSun, 02 Oct 2011 11:19:00 +0200tag:fa.bianp.net,2011-10-02:/blog/2011/scikit-learn-09/General, scikit-learnReworked example gallery for scikit-learnhttp://fa.bianp.net/blog/2011/reworked-example-gallery-for-scikit-learn/<p>I've been working lately in improving the scikit-learn example gallery to show also a small thumbnail of the plotted result. Here is what the gallery looks like now:</p> <img alt="" src="http://fa.bianp.net/blog/static/uploads/2011/09/screenshot.png" /> <p>And the real thing should be already displayed in the <a class="reference external" href="http://scikit-learn.sourceforge.net/dev/auto_examples/index.html">development-documentation</a>. The next thing is to add a static image to those …</p>Fabian PedregosaSun, 04 Sep 2011 20:09:00 +0200tag:fa.bianp.net,2011-09-04:/blog/2011/reworked-example-gallery-for-scikit-learn/scikit-learnscikit-learn’s EuroScipy 2011 coding sprint -- day twohttp://fa.bianp.net/blog/2011/scikit-learns-euroscipy-2011-coding-sprint-day-two/<p><img alt="image0" src="http://fseoane.net/blog/static/uploads/2011/08/all-300x225.jpg" /></p> <p>Today's coding sprint was a bit more crowded, with some notable scipy hackers such as Ralph Gommers, <a class="reference external" href="http://mentat.za.net/">Stefan van der Walt</a>, <a class="reference external" href="http://cournape.wordpress.com/">David Cournapeau</a> or <a class="reference external" href="http://blog.fperez.org/">Fernando Perez</a> from Ipython joining in. On what got done: - We merged <a class="reference external" href="http://www.astro.washington.edu/users/vanderplas/">Jake</a>'s new BallTree code. This is a pure Cython implementation of a nearest-neighbor …</p>Fabian PedregosaThu, 25 Aug 2011 00:33:00 +0200tag:fa.bianp.net,2011-08-25:/blog/2011/scikit-learns-euroscipy-2011-coding-sprint-day-two/General, Python, scikit-learnscikit-learn EuroScipy 2011 coding sprint -- day onehttp://fa.bianp.net/blog/2011/scikit-learn-euroscipy-2011-coding-sprint-day-one/<p>As a warm-up for the upcoming <a class="reference external" href="http://www.euroscipy.org/conference/euroscipy2011">EuroScipy-conference</a>, some of the <a class="reference external" href="http://scikit-learn.sf.net">scikit-learn</a> developers decided to gather and work together for a couple of days. Today was the first day and there was only a handfull of us, as the real kickoff is expected tomorrow. Some interesting coding happened, although most of …</p>Fabian PedregosaTue, 23 Aug 2011 21:38:00 +0200tag:fa.bianp.net,2011-08-23:/blog/2011/scikit-learn-euroscipy-2011-coding-sprint-day-one/miscscikit-learnpythonRidge regression pathhttp://fa.bianp.net/blog/2011/ridge-regression-path/<p>Ridge coefficients for multiple values of the regularization parameter can be elegantly computed by updating the <em>thin</em> SVD decomposition of the design matrix:</p> <div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span> <span class="kn">from</span> <span class="nn">scipy</span> <span class="kn">import</span> <span class="n">linalg</span> <span class="k">def</span> <span class="nf">ridge</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">alphas</span><span class="p">):</span> <span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> <span class="sd"> Return coefficients for regularized least squares</span> <span class="sd"> min ||A x - b||^2 + alpha ||x||^2 …</span></pre></div>Fabian PedregosaTue, 12 Jul 2011 09:21:00 +0200tag:fa.bianp.net,2011-07-12:/blog/2011/ridge-regression-path/miscscikit-learnscipylinear algebraLLE comes in different flavourshttp://fa.bianp.net/blog/2011/lle-comes-in-different-flavours/<p>I haven't worked in the manifold module since <a class="reference external" href="http://fa.bianp.net/blog/2011/manifold-learning-in-scikit-learn/">last time</a>, yet thanks to <a class="reference external" href="http://www.astro.washington.edu/users/vanderplas/">Jake VanderPlas</a> there are some cool features I can talk about. First of, the ARPACK backend is finally working and gives factor one speedup over the <a class="reference external" href="http://fa.bianp.net/blog/2011/locally-linear-embedding-and-sparse-eigensolvers/">lobcpg + PyAMG approach</a>. The key is to use ARPACK's shift-invert mode …</p>Fabian PedregosaThu, 30 Jun 2011 16:22:00 +0200tag:fa.bianp.net,2011-06-30:/blog/2011/lle-comes-in-different-flavours/General, manifold learning, scikit-learnManifold learning in scikit-learnhttp://fa.bianp.net/blog/2011/manifold-learning-in-scikit-learn/<p>The manifold module in <a class="reference external" href="http://scikit-learn.sf.net">scikit-learn</a> is slowly progressing: the <a class="reference external" href="http://fa.bianp.net/blog/2011/locally-linear-embedding-and-sparse-eigensolvers/">locally linear embedding</a> implementation was finally merged along with <a class="reference external" href="http://scikit-learn.sourceforge.net/dev/modules/manifold.html">some documentation</a>. At about the same time but in a different timezone, <a class="reference external" href="http://www.astro.washington.edu/users/vanderplas/">Jake VanderPlas</a> began coding <a class="reference external" href="https://github.com/jakevdp/scikit-learn/compare/master...manifold">other manifold learning methods</a> and back in Paris <a class="reference external" href="http://twitter.com/ogrisel">Olivier Grisel</a> made <a class="reference external" href="http://fa.bianp.net/blog/2011/handwritten-digits-and-locally-linear-embedding/">my digits example</a> a <a class="reference external" href="http://scikit-learn.sourceforge.net/dev/auto_examples/manifold/plot_lle_digits.html">lot …</a></p>Fabian PedregosaTue, 07 Jun 2011 09:19:00 +0200tag:fa.bianp.net,2011-06-07:/blog/2011/manifold-learning-in-scikit-learn/scikit-learnHandwritten digits and Locally Linear Embeddinghttp://fa.bianp.net/blog/2011/handwritten-digits-and-locally-linear-embedding/<p>I decided to test my <a class="reference external" href="http://fa.bianp.net/blog/2011/locally-linear-embedding-and-sparse-eigensolvers/">new Locally Linear Embedding (LLE)</a> implementation against a real dataset. At first I didn't think this would turn out very well, since LLE seems to be somewhat fragile, yielding largely different results for small differences in parameters such as number of neighbors or tolerance, but …</p>Fabian PedregosaWed, 04 May 2011 10:46:00 +0200tag:fa.bianp.net,2011-05-04:/blog/2011/handwritten-digits-and-locally-linear-embedding/General, Python, scikit-learnLow-level routines for Support Vector Machineshttp://fa.bianp.net/blog/2011/low-level-routines-for-support-vector-machines/<p>I've been working lately in improving the low-level API of the libsvm bindings in scikit-learn. The goal is to provide an API that encourages an efficient use of these libraries for expert users. These are methods that have lower overhead than the <a class="reference external" href="http://scikit-learn.sourceforge.net/modules/svm.html">object-oriented interface</a> as they are closer to the …</p>Fabian PedregosaWed, 27 Apr 2011 15:27:00 +0200tag:fa.bianp.net,2011-04-27:/blog/2011/low-level-routines-for-support-vector-machines/General, Python, scikit-learnnew get_blas_funcs in scipy.linalghttp://fa.bianp.net/blog/2011/new-get_blas_funcs-in-scipylinalg/<p>Today got merged some changes I made to function scipy.linalg.get_blas_funcs(). The main enhacement is that get_blas_funcs() now also accepts a single string as input parameter and a dtype, so that fetching the BLAS function for a specific type becomes more natural. For example, fetching the gemm routine for …</p>Fabian PedregosaSat, 23 Apr 2011 18:24:00 +0200tag:fa.bianp.net,2011-04-23:/blog/2011/new-get_blas_funcs-in-scipylinalg/General, Python, scipyLocally linear embedding and sparse eigensolvershttp://fa.bianp.net/blog/2011/locally-linear-embedding-and-sparse-eigensolvers/<p>I've been working for some time on implementing a <a class="reference external" href="http://www.cs.nyu.edu/~roweis/lle/algorithm.html">locally linear embedding</a> algorithm for the upcoming manifold module in scikit-learn. While several implementations of this algorithm exist in Python, as far as I know none of them is able to use a sparse eigensolver in the last step of the …</p>Fabian PedregosaThu, 21 Apr 2011 14:28:00 +0200tag:fa.bianp.net,2011-04-21:/blog/2011/locally-linear-embedding-and-sparse-eigensolvers/General, Python, scikit-learnscikits.learn is now part of pythonxyhttp://fa.bianp.net/blog/2011/scikitslearn-is-now-part-of-pythonxy/<p>The guys behind <a class="reference external" href="http://www.pythonxy.com/">pythonxy</a> have been kind enough to add the latest scikit-learn as an <a class="reference external" href="http://code.google.com/p/pythonxy/wiki/AdditionalPlugins">additional plugin</a> for their distribution. Having scikit-learn being in both <a class="reference external" href="http://www.pythonxy.com/">pythonxy</a> and <a class="reference external" href="http://www.enthought.com/products/epd.php">EPD</a> will hopefully make it easier to use for Windows users. <img alt="pythonxy-logo" src="http://fa.bianp.net/blog/static/uploads/2011/04/pythonxy-logo.png" /> For now I will continue to make windows precompiled binaries, but pythonxy …</p>Fabian PedregosaWed, 20 Apr 2011 13:48:00 +0200tag:fa.bianp.net,2011-04-20:/blog/2011/scikitslearn-is-now-part-of-pythonxy/General, Python, scikit-learnLeast squares with equality constrainhttp://fa.bianp.net/blog/2011/least-squares-with-equality-constrain/<p>The following algorithm computes the Least squares solution || Ax - b|| subject to the equality constrain Bx = d. It's a classic algorithm that can be implemented only using a QR decomposition and a least squares solver. This implementation uses numpy and scipy. It makes use of the new linalg.solve_triangular function …</p>Fabian PedregosaThu, 14 Apr 2011 10:02:00 +0200tag:fa.bianp.net,2011-04-14:/blog/2011/least-squares-with-equality-constrain/Python, Tecnologí­aA profiler for Python extensionshttp://fa.bianp.net/blog/2011/a-profiler-for-python-extensions/<p>Profiling Python extensions has not been a pleasant experience for me, so I made my own package to do the job. Existing alternatives were either hard to use, forcing you to recompile with custom flags like gprofile or desperately slow like valgrind/callgrind. The package I'll talk about is called …</p>Fabian PedregosaWed, 06 Apr 2011 14:02:00 +0200tag:fa.bianp.net,2011-04-06:/blog/2011/a-profiler-for-python-extensions/General, Pythonscikit-learn coding sprint in Parishttp://fa.bianp.net/blog/2011/scikit-learn-coding-sprint-in-paris/<p>Yesterday was the scikit-learn coding sprint in Paris. It was great to meet with old developers (Vincent Michel) and new ones: some of whom I was already familiar with from the mailing list while others came just to say hi and get familiar with the code. It was really great …</p>Fabian PedregosaSat, 02 Apr 2011 12:07:00 +0200tag:fa.bianp.net,2011-04-02:/blog/2011/scikit-learn-coding-sprint-in-paris/General, scikit-learnpy3k in scikit-learnhttp://fa.bianp.net/blog/2011/py3k-in-scikit-learn/<p>One thing I'd really like to see done in <a class="reference external" href="http://gael-varoquaux.info/blog/?p=149">this Friday's scikit-learn sprint</a> is to have full support for Python 3. There's <a class="reference external" href="http://github.com/fabianp/scikit-learn/compare/master...py3k">a branch were the hard word has been done</a> (porting C extensions, automatic 2to3 conversion, etc.), although joblib still has some bugs and no one has attempted to …</p>Fabian PedregosaMon, 28 Mar 2011 15:23:00 +0200tag:fa.bianp.net,2011-03-28:/blog/2011/py3k-in-scikit-learn/GeneralComputing the vector normhttp://fa.bianp.net/blog/2011/computing-the-vector-norm/<p><strong>Update: a fast and stable norm was added to scipy.linalg in August 2011 and will be available in scipy 0.10</strong> Last week I discussed with <a class="reference external" href="http://gael-varoquaux.info/blog/">Gael</a> how we should compute the euclidean norm of a vector a using SciPy. Two approaches suggest themselves, either calling scipy.linalg.norm …</p>Fabian PedregosaTue, 15 Feb 2011 10:31:00 +0100tag:fa.bianp.net,2011-02-15:/blog/2011/computing-the-vector-norm/misclinear algebranormscipySmells like hacker spirithttp://fa.bianp.net/blog/2011/smells-like-hacker-spirit/<p>I was last weekend in <a class="reference external" href="http://fosdem.org/2011/">FOSDEM</a> presenting <a class="reference external" href="http://scikit-learn.sf.net">scikits.learn</a> (<a class="reference external" href="http://fa.bianp.net/talks/fosdem-skl/">here are the slides</a> I used at the Data Analytics Devroom). Kudos to <a class="reference external" href="http://twitter.com/#!/ogrisel">Olivier Grisel</a> and all the people who organized such a fun and authentic meeting!</p> <p><img alt="image0" src="http://farm6.static.flickr.com/5136/5417861859_8480c65eed_m.jpg" /></p> <p><img alt="image1" src="http://farm6.static.flickr.com/5294/5425114531_6eec316967_m.jpg" /></p> Fabian PedregosaFri, 11 Feb 2011 09:50:00 +0100tag:fa.bianp.net,2011-02-11:/blog/2011/smells-like-hacker-spirit/miscpythonsklearnNew examples in scikits.learn 0.6http://fa.bianp.net/blog/2010/new-examples-in-scikitslearn-06/<p>Latest release of <a class="reference external" href="http://scikit-learn.sf.net">scikits.learn</a> comes with an <a class="reference external" href="http://scikit-learn.sourceforge.net/0.6/auto_examples/index.html">awesome collection of examples</a>. These are some of my favorites:</p> <div class="section" id="faces-recognition"> <h2>Faces recognition</h2> <p><a class="reference external" href="http://scikit-learn.sourceforge.net/0.6/auto_examples/applications/plot_face_recognition.html">This example</a> by <a class="reference external" href="http://twitter.com/ogrisel/">Olivier Grisel</a>, downloads a 58MB faces dataset from <a class="reference external" href="http://vis-www.cs.umass.edu/lfw/">Labeled Faces in the Wild</a>, and is able to perform PCA for feature extraction and SVC for classification, yielding …</p></div>Fabian PedregosaFri, 31 Dec 2010 13:55:00 +0100tag:fa.bianp.net,2010-12-31:/blog/2010/new-examples-in-scikitslearn-06/General, scikit-learn, Tecnologí­aWeighted samples for SVMshttp://fa.bianp.net/blog/2010/weighted-samples-for-svms/<p>Based on the work of <a class="reference external" href="http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/#weights_for_data_instances">libsvm-dense</a> by Ming-Wei Chang, Hsuan-Tien Lin, Ming-Hen Tsai, Chia-Hua Ho and Hsiang-Fu Yu I patched the libsvm distribution shipped with scikits.learn to allow setting weights for individual instances. The motivation behind this is to be able force a classifier to focus its attention in …</p>Fabian PedregosaMon, 29 Nov 2010 13:20:00 +0100tag:fa.bianp.net,2010-11-29:/blog/2010/weighted-samples-for-svms/sklearn, pythonComing soon ...http://fa.bianp.net/blog/2010/coming-soon/<img alt="" src="http://farm5.static.flickr.com/4107/5203822436_41b9c350c2.jpg" /> <p>Highlights for this release: * New <a class="reference external" href="http://scikit-learn.sourceforge.net/modules/sgd.html">stochastic gradient descent module</a> by <a class="reference external" href="http://sites.google.com/site/peterprettenhofer/">Peter Prettenhofer</a> * Improved svm module: memory efficiency, automatic class weights. * Wrap for liblinear's Multi-class SVC (option multi_class in <a class="reference external" href="http://scikit-learn.sourceforge.net/modules/generated/scikits.learn.svm.LinearSVC.html">LinearSVC</a>) * New features and performance improvements of text feature extraction. * Improved sparse matrix support, both in main classes (GridSearch) as in sparse …</p>Fabian PedregosaWed, 24 Nov 2010 10:39:00 +0100tag:fa.bianp.net,2010-11-24:/blog/2010/coming-soon/scikit-learn, Tecnologí­amemory efficient bindigs for libsvmhttp://fa.bianp.net/blog/2010/memory-efficient-bindigs-for-libsvm/<p><a class="reference external" href="http://scikit-learn.sf.net">scikits.learn.svm</a> now uses <a class="reference external" href="http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/#libsvm_for_dense_data">LibSVM-dense</a> instead of <a class="reference external" href="http://www.csie.ntu.edu.tw/~cjlin/libsvm/">LibSVM</a> for some support vector machine related algorithms when input is a dense matrix. As a result most of the copies associated with argument passing are avoided, giving 50% less memory footprint and several times less than the python bindings that ship …</p>Fabian PedregosaFri, 19 Nov 2010 15:08:00 +0100tag:fa.bianp.net,2010-11-19:/blog/2010/memory-efficient-bindigs-for-libsvm/General, scikit-learnsolve triangular matrices using scipy.linalghttp://fa.bianp.net/blog/2010/solve-triangular-matrices-using-scipylinalg/<p>For some time now I've been missing a function in scipy that exploits the triangular structure of a matrix to efficiently solve the associated system, so I decided to <a class="reference external" href="http://projects.scipy.org/scipy/changeset/6844">implement it</a> by binding the LAPACK method &quot;trtrs&quot;, which also checks for singularities and is capable handling several right-hand sides. Contrary …</p>Fabian PedregosaSat, 30 Oct 2010 01:13:00 +0200tag:fa.bianp.net,2010-10-30:/blog/2010/solve-triangular-matrices-using-scipylinalg/scipy, Tecnologí­aLARS algorithmhttp://fa.bianp.net/blog/2010/lars-algorithm/<p>I've been working lately with <a class="reference external" href="http://www-sop.inria.fr/members/Alexandre.Gramfort/">Alexandre Gramfort</a> coding the <a class="reference external" href="http://scikit-learn.sf.net/modules/glm.html#lars-algorithm-and-its-variants">LARS algorithm</a> in <a class="reference external" href="hhtp://scikit-learn.sf.net">scikits.learn</a>. This algorithm computes the solution to several general linear models used in machine learning: LAR, Lasso, Elasticnet and Forward Stagewise. Unlike the implementation by coordinate descent, the LARS algorithm gives the full coefficient path along the …</p>Fabian PedregosaThu, 30 Sep 2010 16:01:00 +0200tag:fa.bianp.net,2010-09-30:/blog/2010/lars-algorithm/miscscikit-learnsparseSecond scikits.learn coding sprinthttp://fa.bianp.net/blog/2010/second-scikitslearn-coding-sprint/<p>Las week took place in Paris the second <a class="reference external" href="http://scikit-learn.sf.net">scikits.learn</a> sprint. It was two days of insane activity (115 commits, 6 branches, 33 coffees) in which we did a lot of work, both implementing new algorithms and fixing or improving old ones. This includes: * sparse version of Lasso by coordinate …</p>Fabian PedregosaSun, 12 Sep 2010 22:31:00 +0200tag:fa.bianp.net,2010-09-12:/blog/2010/second-scikitslearn-coding-sprint/scikit-learnSupport for sparse matrices in scikits.learnhttp://fa.bianp.net/blog/2010/support-for-sparse-matrices-in-scikitslearn/<p>I recently added support for sparse matrices (as defined in scipy.sparse) in some classifiers of <a class="reference external" href="http://scikit-learn.sf.net">scikits.learn</a>. In those classes, the fit method will perform the algorithm without converting to a dense representation and will also store parameters in an efficient format. Right now, the only classese that implements …</p>Fabian PedregosaMon, 23 Aug 2010 17:47:00 +0200tag:fa.bianp.net,2010-08-23:/blog/2010/support-for-sparse-matrices-in-scikitslearn/GeneralFlags to debug python C extensions.http://fa.bianp.net/blog/2010/flags-to-debug-python-c-extensions/<p>I often find myself debugging python C extensions from gdb, but usually some variables are hidden because aggressive optimizations that distutils sets by default. What I did not know, is that you can prevent those optimizations by passing flags -O0 -fno-inline to gcc in keyword extra_compile_args (note: this will only …</p>Fabian PedregosaWed, 18 Aug 2010 13:40:00 +0200tag:fa.bianp.net,2010-08-18:/blog/2010/flags-to-debug-python-c-extensions/GeneralJuly in Parishttp://fa.bianp.net/blog/2010/july-in-paris/<p>One of the best things of spending summer in Paris: its parcs (here, with friends &#64; Parc Montsouris).</p> <p><img alt="image0" src="http://farm5.static.flickr.com/4103/4842146900_953f961d64.jpg" /></p> Fabian PedregosaFri, 30 Jul 2010 00:11:00 +0200tag:fa.bianp.net,2010-07-30:/blog/2010/july-in-paris/GeneralSupport Vector machines with custom kernels using scikits.learnhttp://fa.bianp.net/blog/2010/support-vector-machines-with-custom-kernels-using-scikitslearn/<p>It is now possible (using the development version as of may 2010) to use Support Vector Machines with custom kernels in scikits.learn. How to use it couldn't be more simple: you just pass a callable (the kernel) to the class constructor). For example, a linear kernel would be implemented …</p>Fabian PedregosaThu, 27 May 2010 10:42:00 +0200tag:fa.bianp.net,2010-05-27:/blog/2010/support-vector-machines-with-custom-kernels-using-scikitslearn/General, scikit-learn, Tecnologí­aHowto link against system-wide BLAS library using numpy.distutilshttp://fa.bianp.net/blog/2010/howto-link-against-system-wide-blas-library-using-numpydistutils/<p>If your numpy installation uses system-wide BLAS libraries (this will most likely be the case unless you installed it through prebuilt windows binaries), you can retrieve this information at compile time to link python modules to BLAS. The function get_info in numpy.distutils.system_info will return a dictionary that contains …</p>Fabian PedregosaThu, 22 Apr 2010 14:28:00 +0200tag:fa.bianp.net,2010-04-22:/blog/2010/howto-link-against-system-wide-blas-library-using-numpydistutils/Generalscikits.learn 0.2 releasehttp://fa.bianp.net/blog/2010/scikitslearn-02-release/<p>Today I released a new version of the <a class="reference external" href="http://scikit-learn.sourceforge.net">scikits.learn</a> library for machine learning. This new release includes the new libsvm bindings, Jake VanderPlas' BallTree algorithm for *fast* nearest neighbor queries in high dimension, etc. <a class="reference external" href="http://sourceforge.net/mailarchive/message.php?msg_name=4BA72BE3.1010208%40inria.fr">Here</a> is the official announcement. As usual, it can be downloaded from <a class="reference external" href="http://sourceforge.net/projects/scikit-learn/files">sourceforge</a> or from …</p>Fabian PedregosaMon, 22 Mar 2010 11:37:00 +0100tag:fa.bianp.net,2010-03-22:/blog/2010/scikitslearn-02-release/GeneralPlot the maximum margin hyperplane with scikits.learnhttp://fa.bianp.net/blog/2010/plot-the-maximum-margin-hyperplane-with-scikitslearn/<p>Suppose some given data points each belong to one of two classes, and the goal is to decide which class a new data point will be in. In the case of support vector machines, a data point is viewed as a p-dimensional vector (2-dimensional in this example), and we want …</p>Fabian PedregosaWed, 17 Mar 2010 12:24:00 +0100tag:fa.bianp.net,2010-03-17:/blog/2010/plot-the-maximum-margin-hyperplane-with-scikitslearn/General, scikit-learn, Tecnologí­aFast bindings for LibSVM in scikits.learnhttp://fa.bianp.net/blog/2010/fast-bindings-for-libsvm-in-scikitslearn/<p><a class="reference external" href="http://www.csie.ntu.edu.tw/~cjlin/libsvm/">LibSVM</a> is a C++ library that implements several Support Vector Machine algorithms that are commonly used in machine learning. It is a fast library that has no dependencies and most machine learning frameworks bind it in some way or another. LibSVM comes with a Python interface written in swig, but …</p>Fabian PedregosaTue, 09 Mar 2010 15:49:00 +0100tag:fa.bianp.net,2010-03-09:/blog/2010/fast-bindings-for-libsvm-in-scikitslearn/General, scikit-learn, Tecnologí­ascikits.learn coding sprint in Parishttp://fa.bianp.net/blog/2010/scikitslearn-coding-sprint-in-paris/<p>Yesterday we had an extremely productive coding sprint for the <a class="reference external" href="http://scikit-learn.sf.net">scikits.learn</a>. The idea was to put people with common interests in a room and make them work in a single codebase. Alexandre Gramfort and Olivier Grisel worked on <a class="reference external" href="http://scikit-learn.svn.sourceforge.net/viewvc/scikit-learn/trunk/scikits/learn/glm/">GLMNet</a>, Bertrand Thirion and Gaël Varoquaux worked on <a class="reference external" href="http://scikit-learn.svn.sourceforge.net/viewvc/scikit-learn/trunk/scikits/learn/feature_selection/">univariate feature selection …</a></p>Fabian PedregosaThu, 04 Mar 2010 11:25:00 +0100tag:fa.bianp.net,2010-03-04:/blog/2010/scikitslearn-coding-sprint-in-paris/scikit-learn, Tecnologí­aScikit-learn 0.1http://fa.bianp.net/blog/2010/scikit-learn-01/<p>Today I released the first public version of <a class="reference external" href="http://scikit-learn.sourceforge.net">Scikit-Learn</a> (<a class="reference external" href="https://sourceforge.net/mailarchive/message.php?msg_name=4B66D190.5090100%40inria.fr">release notes</a>). It's a python module implementing some machine learning algorithms, and it's shaping quite good.</p> <p>For this release I did not want to do any incompatible changes, so most of them are just bug fixes and updates. For the next …</p>Fabian PedregosaMon, 01 Feb 2010 15:32:00 +0100tag:fa.bianp.net,2010-02-01:/blog/2010/scikit-learn-01/Software, scikit-learnscikit-learn project on sourceforgehttp://fa.bianp.net/blog/2010/scikit-learn-project-on-sourceforge/<p>This week we created a <a class="reference external" href="https://sourceforge.net/projects/scikit-learn/">sourceforge project</a> to host our development of scikit-learn. Although the project already had a directory in scipy's repo, we needed more flexibility in the user management and in the mailing list creation, so we opted for SourceForge. To be honest, after using git and Google …</p>Fabian PedregosaThu, 07 Jan 2010 15:17:00 +0100tag:fa.bianp.net,2010-01-07:/blog/2010/scikit-learn-project-on-sourceforge/General, scikit-learn, Tecnologí­aAfter holidayshttp://fa.bianp.net/blog/2010/after-holidays/<p>New job, new code, new city, new colleagues. Feels something like this:</p> <img alt="" src="http://farm5.static.flickr.com/4027/4240407852_6f461f3776_m.jpg" /> Fabian PedregosaTue, 05 Jan 2010 10:56:00 +0100tag:fa.bianp.net,2010-01-05:/blog/2010/after-holidays/GeneralWinter in Paris is not funnyhttp://fa.bianp.net/blog/2009/winter-in-paris-is-not-funny/<p>This week I arrived to the place where I will be working the following two years: Neurospin.</p> <img alt="" src="http://farm5.static.flickr.com/4042/4206517312_e35b7fa55d_m.jpg" /> <p>It's a research center located 20 km from Paris, and so far things are going smoothly: the place is beautiful, work is great and food is excellent. Well OK, I do miss some …</p>Fabian PedregosaTue, 22 Dec 2009 19:36:00 +0100tag:fa.bianp.net,2009-12-22:/blog/2009/winter-in-paris-is-not-funny/General, Tecnologí­aLast days in Granadahttp://fa.bianp.net/blog/2009/last-days-in-granada/<p>Nice thing about winter in Granada is, that even in the coldest days, the sky is always blue.</p> <img alt="" src="http://farm3.static.flickr.com/2539/4180997669_aa5a45a949_m.jpg" /> Fabian PedregosaTue, 15 Dec 2009 23:42:00 +0100tag:fa.bianp.net,2009-12-15:/blog/2009/last-days-in-granada/Fotos, GeneralLearning, Machine Learninghttp://fa.bianp.net/blog/2009/learning-machine-learning/<p>My new job is about managing an open source package for machine learning in Python. I've had some experience with Python now, but I am a total newbie in the field of machine learning, so my first task will be to find a good reference book in the subject and …</p>Fabian PedregosaTue, 15 Dec 2009 23:34:00 +0100tag:fa.bianp.net,2009-12-15:/blog/2009/learning-machine-learning/General, Tecnologí­aMoving to Paris!http://fa.bianp.net/blog/2009/moving-to-paris/<p>I'm extremely glad that finally I am moving to Paris to work as part of the INRIA crew. I'll be working with <a class="reference external" href="http://gael-varoquaux.info/">Gael Varoquaux</a> and his team in an extremely cool Python related project (more to come on this in the following weeks). Granada has been a great place for …</p>Fabian PedregosaSat, 12 Dec 2009 02:07:00 +0100tag:fa.bianp.net,2009-12-12:/blog/2009/moving-to-paris/GeneralSummer of Code is overhttp://fa.bianp.net/blog/2009/summer-of-code-is-over/<p>Google Summer of Code program is officially over. It has been four months of intense work, exciting benchmarks and patch reviewing. It was a huge pleasure working with you guys! As for the project, I implemented a complete logic module and then an assumption system for sympy (sympy.logic, sympy …</p>Fabian PedregosaSat, 05 Sep 2009 12:21:00 +0200tag:fa.bianp.net,2009-09-05:/blog/2009/summer-of-code-is-over/General, sympy, Tecnologí­aSpeed improvements for ask() (sympy.queries.ask)http://fa.bianp.net/blog/2009/speed-improvements-for-ask-sympyqueriesask/<p>I managed to overcome the overhead in ask() that arises when converting between symbol and integer representation of sentences in conjunctive normal. The result went beyond what I expected. The test suite for the query module got 10x times faster in my laptop. From 26 seconds, it descended to an …</p>Fabian PedregosaThu, 20 Aug 2009 00:36:00 +0200tag:fa.bianp.net,2009-08-20:/blog/2009/speed-improvements-for-ask-sympyqueriesask/General, sympy, Tecnologí­aLogic module (sympy.logic): improving speedhttp://fa.bianp.net/blog/2009/logic-module-sympylogic-improving-speed/<p>Today I've been doing some speed improvements for the logic module. More precisely, I implemented an efficient internal representation for clauses in conjunctive normal form. In practice this means a huge performance boost for all problems that make use the function satisfiable() or dpll_satisfiable(). For example, test_dimacs.py has moved …</p>Fabian PedregosaTue, 18 Aug 2009 23:35:00 +0200tag:fa.bianp.net,2009-08-18:/blog/2009/logic-module-sympylogic-improving-speed/General, sympy, Tecnologí­aRefine modulehttp://fa.bianp.net/blog/2009/refine-module/<p><a class="reference external" href="http://git.sympy.org/?p=sympy.git;a=commit;h=dd679c2751ac0900c47302fd6187ae9eea60918f">This</a> commit introduced a new module in sympy: the refine module. The purpose of this module is to simplify expressions when they are bound to assumptions. For example, if you know that x&gt;0, then you can simplify abs(x) to x. This code was traditionally embedded into the core …</p>Fabian PedregosaMon, 17 Aug 2009 19:20:00 +0200tag:fa.bianp.net,2009-08-17:/blog/2009/refine-module/sympy, Tecnologí­aQuery module - finally in trunkhttp://fa.bianp.net/blog/2009/query-module-finally-in-trunk/<p>The query module is finally in the main SymPy repository. I made substantial changes since last post, most of them at the user interface level (thanks to Vinzent and Mateusz for many insightful comments). Main function is ask(), which replaces the old expression.is_* syntax. You can ask many things …</p>Fabian PedregosaMon, 10 Aug 2009 21:51:00 +0200tag:fa.bianp.net,2009-08-10:/blog/2009/query-module-finally-in-trunk/General, sympy, Tecnologí­adjango, change language settings dynamicallyhttp://fa.bianp.net/blog/2009/django-change-language-settings-dynamically/<p>After some failed attempts, I just found how to change the language settings dynamically in django, and I thought it could be useful to someone. Just use function activate() from django.utils.translation. For example: [cc lang=&quot;python&quot;] from django.utils.translation import activate activate('es-ES') [/cc] will change global …</p>Fabian PedregosaFri, 07 Aug 2009 16:12:00 +0200tag:fa.bianp.net,2009-08-07:/blog/2009/django-change-language-settings-dynamically/General, Tecnologí­acan we merge now, pleeease ?http://fa.bianp.net/blog/2009/can-we-merge-now-pleeease/<p>Three months after I began to write sympy.queries, I feel it's about time to include it in sympy's trunk, so today I sent for review <a class="reference external" href="http://groups.google.com/group/sympy-patches/browse_thread/thread/76dcdfd0994a1c81">4 patches that implement the complete query module</a>. It's been a lot of fun, but it has also caused me some headaches ... specially last …</p>Fabian PedregosaTue, 21 Jul 2009 22:36:00 +0200tag:fa.bianp.net,2009-07-21:/blog/2009/can-we-merge-now-pleeease/General, sympy, Tecnologí­aRefine module, proof of concepthttp://fa.bianp.net/blog/2009/refine-module-proof-of-concept/<p>The 0.6.5 release of SymPy is taking longer than expected because <a class="reference external" href="http://code.google.com/p/sympy/issues/detail?id=1521">some bugs in the testing framework</a>, so my query module is not merged into trunk (yet). In the meantime, I am implementing a refine module (very little code is available yet). The refine module implements a refine …</p>Fabian PedregosaThu, 09 Jul 2009 02:49:00 +0200tag:fa.bianp.net,2009-07-09:/blog/2009/refine-module-proof-of-concept/General, sympy, Tecnologí­aPreparing a new releasehttp://fa.bianp.net/blog/2009/preparing-a-new-release/<p>Last days I've been busy preparing <a class="reference external" href="http://groups.google.com/group/sympy/browse_thread/thread/88474cde3bc6e350#">the first public beta of SymPy 0.6.5</a>. Most of the time was spent solving a bug that made documentation tests fail under python2.4, but now that this is solved, I hope that by the end of the week we could have …</p>Fabian PedregosaTue, 30 Jun 2009 08:57:00 +0200tag:fa.bianp.net,2009-06-30:/blog/2009/preparing-a-new-release/General, sympy, Tecnologí­aEfficient DPLL algorithmhttp://fa.bianp.net/blog/2009/efficient-dpll-algorithm/<p>Background: DPLL is the algorithm behind SymPy's implementation of logic.inference.satisfiable After reading the original papers by Davis &amp; Putnam [1], I managed to implement a more efficient version of the DPLL algorithm. It is 10x times faster on medium-sized problems (40 variables), and solves some wrong result bugs [2 …</p>Fabian PedregosaSun, 28 Jun 2009 18:16:00 +0200tag:fa.bianp.net,2009-06-28:/blog/2009/efficient-dpll-algorithm/GeneralQueries and performancehttp://fa.bianp.net/blog/2009/queries-and-performance/<p>After some hacking on the queries module, I finally got it right without the <a class="reference external" href="http://fa.bianp.net/blog/?p=149">limitations of past versions</a>. You can check it out from my repo <a class="reference external" href="http://fa.bianp.net/git/sympy.git">http://fa.bianp.net/git/sympy.git</a>, branch master. It now relies even more on logic.inference.satisfiable(), which is just an implementation of …</p>Fabian PedregosaTue, 23 Jun 2009 00:02:00 +0200tag:fa.bianp.net,2009-06-23:/blog/2009/queries-and-performance/General, sympy, Tecnologí­aReading CNF fileshttp://fa.bianp.net/blog/2009/reading-cnf-files/<p><p>The DIMACS CNF file format is used to define a Boolean expression, written in conjunctive normal form, that may be used as an example of the satisfiability problem. The new logic module (sympy.logic) can read the content of a cnf file and transform it into a boolean expression suitable …</p></p>Fabian PedregosaSat, 20 Jun 2009 16:27:00 +0200tag:fa.bianp.net,2009-06-20:/blog/2009/reading-cnf-files/General, sympy, Tecnologí­aLogic module mergedhttp://fa.bianp.net/blog/2009/logic-module-merged/<p>Yesterday I finally merged the logic module in sympy's official master branch, and should be released together with SymPy 0.6.5. Next thing to do: profile the code and write some docs before the release.</p> Fabian PedregosaFri, 19 Jun 2009 12:23:00 +0200tag:fa.bianp.net,2009-06-19:/blog/2009/logic-module-merged/General, sympy, Tecnologí­aThe boolean satisfiability problemhttp://fa.bianp.net/blog/2009/the-boolean-satisfiability-problem/<p><p>Most annoying problem in my implementation of the query system is that it will not solve implications if the implicates are far away from each other. For instance, if the graph of known facts is something like this</p> <pre class="literal-block"> Integer ----&gt; Rational --&gt; Real --&gt; Complex ^ ^ | | | ------- | | Prime Even ^ | | MersennePrime </pre> <p>Then it will not know …</p></p>Fabian PedregosaMon, 15 Jun 2009 06:00:00 +0200tag:fa.bianp.net,2009-06-15:/blog/2009/the-boolean-satisfiability-problem/General, sympy, Tecnologí­aInitial implementation of the query systemhttp://fa.bianp.net/blog/2009/initial-implementation-of-the-query-system/<p>I sent <a class="reference external" href="http://groups.google.com/group/sympy-patches/browse_thread/thread/e56ceda0038b7c23">some patches</a> to sympy-patches with an initial implementation of the query system. You can check it out by pulling from my branch: <tt class="docutils literal">git pull <span class="pre">http://fa.bianp.net/git/sympy.git</span> master</tt> into your sympy repo. Some examples of what you can do (sample isympy session): <tt class="docutils literal">In [1 …</tt></p>Fabian PedregosaFri, 12 Jun 2009 06:36:00 +0200tag:fa.bianp.net,2009-06-12:/blog/2009/initial-implementation-of-the-query-system/General, sympy, Tecnologí­aAssumption system and automatic theorem proving. Should I be learning LISP ?http://fa.bianp.net/blog/2009/assumption-system-and-automatic-theorem-proving-should-i-be-learning-lisp/<p>This is the third time I attempt to write the assumption system. Other attempts could be described as me following the rule: “For any complex problem, there is always a solution that is simple, clear, and wrong.” My <a class="reference external" href="http://groups.google.com/group/sympy-patches/browse_thread/thread/b6fd5402e729f58/8006779044c41a17?lnk=gst&amp;q=fabian+assumptions#8006779044c41a17">first attempt</a> (although better than the current assumption system) did use very …</p>Fabian PedregosaWed, 03 Jun 2009 13:50:00 +0200tag:fa.bianp.net,2009-06-03:/blog/2009/assumption-system-and-automatic-theorem-proving-should-i-be-learning-lisp/General, sympy, Tecnologí­aHomenaje a Antonio Vega en La Perchahttp://fa.bianp.net/blog/2009/homenaje-a-antonio-vega-en-la-percha/<p>El pasado jueves estuvimos en La Percha tocando algunas canciones de Antonio Vega. El vídeo se lo ha currado <a class="reference external" href="http://retrovisor.net">mi padre</a> mezclando el sonido del directo con una grabación que hicimos en casa de Migue</p> <p><a class="reference external" href="http://vimeo.com/4926476">LOS ESCLAVOS: homenaje a Antonio Vega</a> from <a class="reference external" href="http://vimeo.com/user938253">Felipe Pedregosa</a> on <a class="reference external" href="http://vimeo.com">Vimeo</a>.</p> </p>Fabian PedregosaMon, 01 Jun 2009 15:34:00 +0200tag:fa.bianp.net,2009-06-01:/blog/2009/homenaje-a-antonio-vega-en-la-percha/Canciones, General, Grupo