Collaborators:  Xin Li, CMU:  Shawn Blanton, CMU;  Duane Boning, MIT;  Emrah Acar, IBM;  Frank Liu, IBM

At the nanoscale, nothing is deterministic.  Every behavior we want to model is a messy smear of correlated probability.  This creates major problems when trying to design modern integrated circuits.  Spatial variation – differences in the behavior of our designs based on where they are, how close they are – is a huge problem.   Things vary at the level of individual transistors, functional blocks, chips, wafer, and lots (different sets of wafers all manufactured together).  Where do we look for methods to attack such problems?   It turns out that Bayesian Statistics, and related methods from Machine Learning (ML) hold the key to building useful predictive models.

We have designed and validated a range of useful methods to deal with spatial variation.  This includes tools for predicting a minimum number of measurement samples from a wafer to predict behaviors at other non-measured locations (“virtual probes”);  tools for predicting where to put those samples for optimal results, in an information theoretic sense;  tools to decompose these variations intodecompose process variation into two different components: (1) spatially correlated variation, and (2) uncorrelated random variation; and tools for automatically clustering the spatial signatures of wafers to aid yield improvement.

Key Papers/Talks:

• Xin Li, Rob A. Rutenbar, R. Shawn Blanton, “Virtual Probe: A Statistically Optimal Framework for Minimum-Cost Silicon Characterization of Nanoscale Integrated Circuits,”  Proc. ACM/IEEE Internaitonal Conference on CAD (ICCAD),  pp. 433-440, November 2009.
• Wangyang Zhang, Xin Li and Rob A. Rutenbar, “Bayesian Virtual Probe: Minimizing variation characterization cost for nanoscale IC technologies via Bayesian inference,” Proc. ACM/IEEE Design Automation Conference (DAC), pp. 262-267, July 2010. (Winner of 2010 DAC Best Paper Award.)
• Wangyang Zhang, Xin Li, Emrah Acar, Frank Liu, Rob A. Rutenbar, “ Multi-Wafer Virtual Probe: Minimum-cost variation characterization by exploring wafer-to-wafer correlation,” Proc. ACM/IEEE International Conference on CAD (ICCAD), pp. 47-54, November 2010.
• W. Zhang, K. Balakrishnan, X. Li, D. Boning, R.A. Rutenbar, “Toward efficient spatial variation decomposition via sparse regression,” Proc ACM/IEEE International Conference on CAD (ICCAD), pp. 162-169, November 2011.
• Wangyang Zhang, Karthik Balakrishnan, Xin Li, Duane Boning, Emrah Acar, Frank Liu and Rob A. Rutenbar, “Spatial Variation Decomposition via Sparse Regression,” Proc .IEEE  International Conference on Integrated Circuit Design & Technology (ICICDT), invited, June 2012.
• Wangyang Zhang, Xin Li, Frank Liu, Emrah Acar,  Rob A. Rutenbar, R. Shawn Blanton, “Virtual Probe: A Statistical Framework for Low-Cost Silicon Characterization of Nanoscale Integrated Circuits,” IEEE Trans. On CAD, Vol. 30, No. 12, pp. 1814 – 1827, December 2011.

Collaborators:  Naresh Shanbhag, Illinois;  Paris Smaragdis, Illinois

Machine learning (ML) technologies have revolutionized the ways in which we interact with large-scale, imperfect, real-world  data.  We can cast these as high-dimensional optimizations;  we can manage the inherent uncertainties via the mechanics of probability; and we can search for answers to complex questions across a range of vital applications. What we cannot do is solve these problems quickly and efficiently.  Data volume, data complexity, data rate, data uncertainty, and data modalities all expand exponentially.   There are today problems that take days to solve, that need to be completed in seconds for timely application; and most of these techniques are entirely outside the feasible power/speed envelop of modern mobile appliances.   If we could accelerate these core computations, we could dramatically increase the scale, and speed, and the universe of applicability of these important algorithms.

This project is working at the intersection of (i) machine learning in hardware, and (ii) stochastic computation in nanoscale silicon.  We are building some of the first large-scale all-hardware implementations of inference methods from the arena of probabilistic graphical models.   We are working currently on applications in machine vision and listening, but planning to move to large-scale data analytics.   We are also applying principled strategies from stochastic computation, in which the inevitable errors in the fundamental device fabric are mitigated in a manner integrated with the application itself. Stochastic computation matches the robustness of the silicon computational fabric to the reliability of the data being processed.   We have excellent early results in vision (TRW-S inference running stereo vision at video frame rates) and listening (a novel graphical model for audio source separation).

Key Papers/Talks

• Jaesik Choi, Abner Guzmán-Rivera and Eyal Amir, “Lifted Relational Kalman Filtering,” in Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI), 2011.
• Jungwook Choi and Rob A. Rutenbar, “Hardware Implementation of MRF MAP Inference on an FPGA Platform,” Proc. 22^nd Intl Conference on Field Programmable Logic and Applications (FPL), August 2012.
• Minje Kim, Paris Smaragdis, Glenn G. Ko and Rob A. Rutenbar, “Stereophonic Spectrogram Segmentation Using Markov Random Fields,” Proc. 2012 IEEE Int’l Workshop on Machine Learning for Signal Processing, to appear, September 2012.
• Dhruv Batra, Payman Yadollahpour, Abner Guzman-Rivera, Gregory Shakhnarovich, “Diverse M-Best Solutions in Markov Random Fields,” in Proc. European Conference on Computer Vision (ECCV) October 2012.
• Abner Guzmán-Rivera, Dhruv Batra, Pushmeet Kohli, “Multiple Choice Learning: Learning to Produce Multiple Structured Outputs,” ? NIPS 2012.

Our new proposed degree program – we’re informally calling it “CS+X” – which is to be offered in the College of Liberal Arts and Sciences, was reviewed in the campus paper today.  This is an exciting new collaboration which will allow degrees in thing like Computational Anthropology and Computational Chemistry.   Lots of the most existing work is at the intersection of science, social science, humanities, etc, and our own IT work.   This new degree will let us target these novel combinations.

Lots of buzz generated by our Chicago After Hours event held on campus this month.  Here is an interview I did with Crain’s, talking about technology connections from UIUC to Chicago:   http://www.chicagobusiness.com/article/20121003/NEWS08/121009920/emanuel-pitches-u-of-i-tech-students-to-some-effect

Lab Researchers: Patrick Bourke, Jeffrey Johnston, Edward C. Lin, Kai Yu

Collaborators: Richard M. Stern, CMU; Tsuhan Chen, CMU

Whether running on a single cell phone, a conventional PC, or an enterprise-level server farm – all of today’s state-of-the-art speech recognizers exist as complex software running on conventional computers. This is profoundly limiting for applications in which speed or mobility are essential. We need recognizers which can run significantly faster than realtime, to search large online media streams for keywords. We need desktop-quality recognizers to evolve off our desktops into the small, power-limited appliances we carry in our pockets. To do this, we must move the core of today’s most successful speech recognition strategies directly into silicon. This is the path taken by critical tasks such as graphics, which have seen performance improvements of six orders of magnitude over the last decade. The CMU “In Silico Vox” project is developing a range of custom architectures for speech recognition. A recent example is our working FPGA-based prototype which handles a 1000-word vocabulary, and is, to the best of our knowledge, the most complex recognizer ever rendered completely in hardware.

Key Papers/Talks

• “The Talking Cure,” The Economist (Technology Quarterly Issue), pp 11, March 12 2005. pdf
• Patrick Bourke, Rob A. Rutenbar, “A High-Performance Hardware Speech Recognition System for Mobile Applications,” Proc Semiconductor Research Corporation TECHCON, Aug. 2005. pdf
• R.A. Rutenbar, E.C. Lin, K. Yu, T. Chen, “In Silico Vox: Toward Speech Recognition in Silicon,” 2006 (Eighteenth) Hot Chips Symposium, August 2006.
  pdf
• Stephen Shankland, “Chips Promise to Boost Speech Recognition,” CNET News.com, Aug 22, 2006.
• Edward C. Lin, Kai Yu, Rob A. Rutenbar and Tsuhan Chen, “Moving Speech Recognition from Software to Silicon: the In Silico Vox Project,” Proc. International Conference on Spoken Language Processing (InterSpeech2006), September 2006. pdf
• Linda Daily Paulson, “Speech Recognition Moves from Software to Hardware,” IEEE Computer, pp 15-18, Nov. 2006.pdf
• Edward C. Lin, Kai Yu, Rob A. Rutenbar and Tsuhan Chen, “A 1000-Word Vocabulary, Speaker- Independent, Continuous Live-Mode Speech Recognizer Implemented in a Single FPGA,” Proc. ACM International Symposium on FPGAs, Feb. 2007. pdf
• Rob A. Rutenbar, “Toward Speech Recognition in Silicon: the Carnegie Mellon In Silico Vox Project,” Invited Talk, 2007 Brammer Memorial Lecture, Wayne State University, Oct 2007. pdf
• K. Yu, R.A. Rutenbar, “Generating Small, Accurate Acoustic Models with a Modified Bayesian Information Criterion,” Proc. Interspeech 2007, August 2007. pdf
• Patrick Bourke, Rob A. Rutenbar, “A Low-Power Hardware Search Architecture for Speech Recognition,” Proc Interspeech’08, Sept. 2008. pdf

Group Researchers: Sonia Singhal

Collaborators: Amith Singhee, IBM; Benton H. Calhoun, U. Virginia; Xin Li, CMU

Moore’s law device scaling dramatically increases the statistical variability with which tomorrow’s chips must contend. Devices with atomic dimensions don’t have deterministic parameters: every behavior we want to model is a messy smear of probability. How should we attack such problems? Is slow, expensive Monte Carlo analysis our only option? Is the silicon community unique in facing such problems? As it turns out, problems in computational finance and risk analysis share many of the characteristics that challenge us in statistical circuit analysis: high dimensionality, profound nonlinearity, stringent accuracy requirements, and expensive analysis (i.e., circuit simulation). This project is adapting computational ideas from Wall Street for use in the silicon world. The same methods used to price complex securities can be adapted to compute silicon yields, giving speedups of 2x – 50x. Methods used to analyze the statistics of rare events (like the size of the biggest wave in a hurricane like Katrina) can be used to analyze failures in SRAM, giving speedups of 20,000x.

Key Papers/Talks:

• Amith Singhee and Rob A. Rutenbar, “From Finance to Flip Flops: A Study of Fast Quasi-Monte Carlo Methods from Computational Finance Applied to Statistical Circuit Analysis,” Proc. IEEE 8th International Symposium on Quality Electronic Design (ISQED), March 2007. pdf
• Amith Singhee and Rob A. Rutenbar, “Statistical Blockade: A Novel Method for Very Fast Money Carlo Simulation of Rare Circuit Events, and its Application ,” Proceedings of the 10th Conference on Design, Automation and Test in Europe (DATE 07), April 2007. Winner, DATE2007 Best Paper. pdf
• Amith Singhee and Rob A. Rutenbar, “Beyond Low-Order Statistical Response Surfaces: Latent Variable Regression for Efficient, Highly Nonlinear Fitting,” Proc. ACM/IEEE Design Automation Conference, June 2007. pdf
• J. Wang, A. Singhee, R.A. Rutenbar, B. H. Calhoun, “Modeling the Minimum Standby Supply Voltage of a Full SRAM Array,” Proc. European Solid State Circuits Conference (ESSCIRC), October 2007. pdf
• R.A. Rutenbar, “From Wall Street to Silicon Valley: Using the Mathematics of Money & Risk for Fast Statistical IC Design ,” Invited Keynote given at 5th Int’l System-on-Chip (SoC) Conference, Nov. 2007. pdf
• A. Singhee, J. Wang, B. H. Calhoun, R. A. Rutenbar, “Recursive Statistical Blockade,” Proc. 2008 International Conference on VLSI, January 2008. Winner, Best Student Paper Award. pdf
• A. Singhee, S. Singhal, R. A. Rutenbar, “Exploiting Correlation Kernels for Efficient Handling of Intra-Die Spatial Correlation, with Application to Statistical Timing,” Proc. Design Automation and Test in Europe Conference (DATE), March 2008. pdf
• Benton H. Calhoun, Yu Cao, Xin Li, Ken Mai, Lawrence T. Pileggi, Rob A. Rutenbar and Kenneth L. Shepard, “Digital Circuit Design Challenges and Opportunities in the Era of Nanoscale CMOS,” Proceedings of the IEEE, Vol. 96, No. 2, pages 343-365, February 2008. pdf
• Rob A. Rutenbar, “From FInance to Flip Flops: Using the Mathematics of Money and Risk to Understand the Statistics of Nanoscale Circuits” , invited talk given at Cadence Design Systems, Oct 2008. pdf
• Amith Singhee, Sonia Singhal, Rob A. Rutenbar, “Practical, Fast Monte Carlo Static Timing Analysis: Why and How,” Proc. ACM/IEEE International Conference on CAD, Nov 2008. pdf
• Rob A. Rutenbar, “From FInance to Flip Flops: Using the Mathematics of Money and Risk to Understand the Statistics of Nanoscale Circuits” , invited talk given at CANDE’08 Workshop, Nov 2008. pdf

Group Researchers: Pragati Tiwary, Saurabh Tiwary, Dong Chen, Senthil Velu

Collaborators: Tamal Mukherjee, CMU; Jeyanandh Paramesh, CMU; Yu-tsun Chien, ITRI, Taiwan

Synthesis tools for cell-level analog building blocks (10-100 devices) have recently made the transition from academic prototypes to commercial products. These tools can help to size, bias, center, optimize and lay out critical analog and RF circuits. Unfortunately, they do not scale directly to system level, where we may have 10-100 blocks, integrated with many digital functions, using problematic scaled devices better suited for NAND gates than opamps. A key obstacle is the essential architecture of the today’s cell-level tools: they employ simulation-based synthesis, and rely on full SPICE level evaluation of each evolving solution candidate, in a large global-optimization framework, distributed across networked workstations. The strategy has been the key enabler for cell-level designs that are ‘trustworthy’ to designers. However, we cannot simulate system-level designs rapidly enough to use this paradigm. Alternatives have been proposed, none successfully. This projects is developing novel synthesis strategies based on three key ideas: (1) we strive to reuse (i.e., to respect) the existing models used by practicing system designers; (2) we optimally leverage existing cell-level optimization tools; (3) we bring first-order statistical optimization into the same framework, by extracting statistical tradeoff models from key circuits, and inserting these into the system-level design problem.

Key Papers/Talks

• S.K. Tiwary, S. Velu, R.A. Rutenbar, T. Mukherjee, “Pareto-Optimal Modeling for Efficient PLL Optimization,” Technical Proceedings of the 2004 NSTI Nanotechnology Conference and Trade Show, 2004. pdf
• Yu-Tsun Chien, Dong Chen, Jea-Hong Lou, Gin-Kou Ma, Rob A. Rutenbar, and Tamal Mukherjee,”Designer-Driven Topology Optimization for Pipelined Analog to Digital Converters,” in Design Automation and Test in Europe (DATE 2005), March 2005, pp. 279-280. pdf
• R.A. Rutenbar, “Design Automation for Analog: The Next Generation of Tool Challenges,” 1st IBM Academy Conference on Analog Design, Technology, Modeling and Tools, IBM T.J. Watson Research Labs, Sept. 2006. pdf
• R.A. Rutenbar, “Design Automation for Analog: The Next Generation of Tool Challenges,” IEEE/ACM International Conference on Computer-Aided Design, 2006. ICCAD ’06. Nov. 2006. pdf
• R.A. Rutenbar, G.G.E. Gielen, J. Roychowdhury, “Hierarchical Modeling, Optimization, and Synthesis for System-Level Analog and RF Designs,” Proceedings of the IEEE, Vol. 95, No. 3, March 1997. pdf
• S. Tiwary, P. Tiwary, R.A. Rutenbar, “Generation of yield-aware Pareto surfaces for hierarchical circuit design space exploration,” Proc ACM/IEEE Design Automation Conf, June 2006. pdf

Collaborators: Gi-Joon Nam, IBM Austin Research Labs; Karem Sakallah, U. Michigan

Boolean Satisfiability (or “SAT”) is the problem of deciding if a large, complex Boolean equation is satisfiable, i.e., if there is any assignment of 0s and 1s to its component variables that renders the overall equation identically “1″. If not, the problem is unsatisfiable. Advances in representations (BDDs) and solvers (SAT) for the problem make it possible to consider formulating some non-Boolean problems into this Boolean form. Our work in this area mainly targets routing problems from the world of FPGAs. Our early paper in ISFPGA97 was the first to concretely pose the problem of discrete FPGA routing as a SAT problem. Subsequent advances in both the formulation and the sophistication on SAT solvers made it possible to target some extremely complex geometric problems, and even some novel general-purpose problems such as deciding which subset of constraints one might need to abandon to reach a partial (“sub-satistfiable”) solution to the problem.

Key Papers/Talks

• R. Glenn Wood and Rob A. Rutenbar, “FPGA routing and routability estimation via Boolean satisfiability,” Proc. ACM International Symposium on FPGAs, Feb. 1997. pdf
• R. Glenn Wood and Rob A. Rutenbar, “FPGA routing and routability estimation via Boolean satisfiability,” IEEE Trans on VLSI Systems, Vol. 6, No. 2, June 1998. pdf
• G.-J. Nam, K. Sakallah and R. Rutenbar, “Satisfiability-based FPGA routing,” Proc. International Conference on VLSI Design, Jan. 1999, Goa, India.
• G.-J. Nam, K. Sakallah and R. Rutenbar, “Satisfiability-based layout revisited: detailed routing of complex FPGAs via search-based boolean SAT,” Proc. ACM International Symposium on Field Programmable Gate Arrays, Feb. 1999.pdf
• G.-J. Nam, K. Sakallah and R. Rutenbar, “A boolean satisfiability-based incremental rerouting approach with application to FPGAs,” Proc. Design Automation and Test in Europe (DATE01), March 2001. pdf
• G.-J. Nam, F. Aloul, K. Sakallah and R. Rutenbar , “A comparative study of two boolean formulations on FPGA detailed routing constraints,” Proc. ACM International Symposium on Physical Design (ISPD01), April, 2001. pdf
• Hui Xu, Rob A. Rutenbar, Karem Sakallah, “sub-SAT: a formulation for relaxed boolean satisfiability with applications in routing,” Proc. ACM International Symposium on Physical Design (ISPD02), April 2002. pdf
• G.-J. Nam, K. Sakallah and R. Rutenbar, “A new FPGA detailed routing approach via search-based boolean satisfiability,” IEEE Transactions on CAD, vol.21, no.6, June 2002. pdf
• G.-J. Nam, K. Sakallah and R. Rutenbar , “Hybrid routing for FPGAs by integrating boolean safisfiability with geometric search,” International Conference on Field Programmable Logic and Applications, Sep. 2002, La Motte, France.
• Hui Xu, Rob A. Rutenbar, Karem Sakallah, “sub-SAT: a formulation for relaxed boolean satisfiability with applications in routing,” IEEE Trans CAD, Vol 22, No 6, June 2003. pdf
• G.-J. Nam, F. Aloul, K. Sakallah and R. Rutenbar , “A comparative study of two boolean formulations on FPGA detailed routing constraints,” IEEE Transactions on Computers, vol. 53, no.6, June 2004. pdf