Category Archives: Notes

张鹏的生活感悟

谷歌定制芯片(TPU)加速机器学习算法

ASIC定制芯片,可以通过优化硬件执行逻辑,来提升芯片的单位瓦特计算效能。从而更少的硬件和能耗,完成更复杂的算法,从而提升整体产品线的竞争力。 谷歌在这个领域并不是先例,微软通过FPGA加速索引和搜索过程。 OSRF(开放机器人联盟)通过FPGA实现对机器手的高速精密控制。 简单讲,专用芯片实现优势是复杂的算法,更低的能耗和更好的执行效率。

谷歌的专有芯片,是完全为TensorFlow机器学习框架定制的。目前服务100多个产品,包括Gmail, 街景,语音搜索,搜索页面质量评价等。 前些时间热门的AlphaGo,也是通过TPU来支持。 不多说了,请看原文。

Machine learning provides the underlying oomph to many of Google’s most-loved applications. In fact, more than 100 teams are currently using machine learning at Google today, from Street View, to Inbox Smart Reply, to voice search. But one thing we know to be true at Google: great software shines brightest with great hardware underneath. That’s why we started a stealthy project at Google several years ago to see what we could accomplish with our own custom accelerators for machine learning applications. The result is called a Tensor Processing Unit (TPU), a custom ASIC we built specifically for machine learning — and tailored for TensorFlow. We’ve been running TPUs inside our data centers for more than a year, and have found them to deliver an order of magnitude better-optimized performance per watt for machine learning. This is roughly equivalent to fast-forwarding technology about seven years into the future (three generations of Moore’s Law). TPU is tailored to machine learning applications, allowing the chip to be more tolerant of reduced computational precision, which means it requires fewer transistors per operation. Because of this, we can squeeze more operations per second into the silicon, use more sophisticated and powerful machine learning models and apply these models more quickly, so users get more intelligent results more rapidly. A board with a TPU fits into a hard disk drive slot in our data center racks.

Tensor Processing Unit board

Tensor Processing Unit board

TPU is an example of how fast we turn research into practice — from first tested silicon, the team had them up and running applications at speed in our data centers within 22 days. TPUs already power many applications at Google, including RankBrain, used to improve the relevancy of search results and Street View, to improve the accuracy and quality of our maps and navigation. AlphaGo was powered by TPUs in the matches against Go world champion, Lee Sedol, enabling it to “think” much faster and look farther ahead between moves.

Server racks with TPUs used in the AlphaGo matches with Lee Sedol

Server racks with TPUs used in the AlphaGo matches with Lee Sedol


Our goal is to lead the industry on machine learning and make that innovation available to our customers. Building TPUs into our infrastructure stack will allow us to bring the power of Google to developers across software like TensorFlow and Cloud Machine Learning with advanced acceleration capabilities. Machine Learning is transforming how developers build intelligent applications that benefit customers and consumers, and we’re excited to see the possibilities come to life.

最近常读的社科类书单

繁荣的真谛

大篇幅讨论了裙带资本主义,公司腐败与自由竞争的关系,如果一个公司形成垄断,更容易变成社会主义公司。因为垄断产生了更多的利润,并且公司经理人面临这更小的外部竞争,从而滋生腐败,降低整体公司的运营效率。

购买链接 http://item.jd.com/11745121.html

关于作者

路易吉·津加莱斯(Luigi Zingales),美国芝加哥大学布斯商学院企业家精神和金融学Robert C. McCormack讲席教授。公司金融和公司治理领域最重要的学者之一,欧洲公司治理委员会研究员,美国国民经济研究局、经济政策研究中心研究员。1992年获美国麻省理工学院经济学博士学位。

卓有成效的管理者

德鲁克经典著作,最早是高翔推荐。这本书最重要的观点,贡献通过组织/个体 与外部的交互产生。另外个体也是自身的管理者,每个人都是管理者。

购买链接 http://item.jd.com/10059507.html

彼得原理

劳伦斯J.彼得在层级组织进行多年调查研究之后发现一个颠覆传统思想的“彼得原理”:在层级组织中,员工倾向于晋升到自身不胜任的职位,其结果是,企业中的每个职位终将由不胜任的员工所占据。

同时,针对大型组织的管理,作者提出了蔓藤式晋升与冲击式晋升两个概念。

    冲击式晋升是假晋升。获得晋升的员工并没有比以前担负更重职责,并没有在新职位上完成多于原先的职位的工作量,有时简直是在制造一大堆无意义、无价值的工作机会和垃圾。冲击式晋升这类的假晋升是造成层级组织机构臃肿、冗员众多、人浮于事的原因之一。蔓藤式晋升和冲击式晋升没有什么本质的区别,都是在种种原因不能解雇不胜任员工的前提下,为不胜任员工制造一些可无的、无比有好且员工看起来他们获得了真正的晋升的职位,以欺蒙他人、隔离冗员、改变重要岗位的不胜任状态。

购买链接 http://item.jd.com/11245245.html

Google 比利时数据中心遭雷击,导致数据丢失

Google loses data as lightning strikes

Google loses data as lightning strikes

 

 

 

From Thursday 13 August 2015 to Monday 17 August 2015,Google 在比利时的数据中心遭到4次雷击,问题初步原因是雷击破坏了供电系统,导致GCE服务( Google Compute Engine ) 有用户丢失数据。突然断电导致GCE的虚拟机不能及时会写数据。看来这个数据中心的UPS系统完成切换,但是仍旧发生了数据丢失。

受影响的数据存储占总存储空间的0.000001%,但是考虑到Google数据中心的海量数据,总的影响范围会在较高水平。

根据Google公开的报告,在事故发生区间,5%的回写IO至少发生一次读写错误。

Google比利时数据中心2007年开始建造,耗资2.5亿欧元,在2010正式投入使用。数据中心环境友好,能效比高,采用水冷系统,参考(advanced evaporative cooling system )。 这个数据 中心创造了3900个就业岗位,估计带给比利时政府22亿欧元的收入。

参考Google事故声明原文,非常值得国内同行学习。

In an online statement, Google said

Nvidia Deep Learning Courses

The focus of Deep Learning is currently on specific perceptual tasks, and there are many successes.

Course Schedule

All classes start at 9am PT and will be recorded for later viewing.

7/22 Class #1 – Introduction to Deep Learning (30 mins + Q&A) – Recording, Slides, Hands-on lab
7/29 Office Hours for Class #1 (1 hour of Q&A)Recording, Slides, Q&A log
8/5 Class #2 – Getting Started with DIGITS interactive training system for image classification (30 mins + Q&A)Recording, SlidesHands-on lab
8/12 Office Hours for Class #2 (1 hour of Q&A)
8/19 Class #3 – Getting Started with the Caffe Framework
8/26 Office Hours for Class #3
9/2 Class #4 – Getting Started with the Theano Framework
9/9 Office Hours for Class #4
9/16 Class #5 – Getting Started with the Torch Framework
9/23 Office Hours for Class #5

Please send your questions to dl-course@nvidia.com before the Office Hours sessions, so the instructors can prepare helpful answers. You might even get a faster response!

The hands-on lab exercises for each class will be available at nvidia.qwiklab.com for free during the course.

More Deep Learning Courses

  • Andrew Ng’s Coursera course provides a good introduction to deep learning (Coursera, YouTube)
  • Yann LeCun’s NYU Course on Deep Learning, Spring 2014 (TechTalks)
  • Geoffrey Hinton’s “Neural Networks for Machine Learning” course from Oct 2012 (Coursera)
  • Rob Fergus’s “Deep Learning for Computer Vision” tutorial from NIPS 2013 (slides, video)
  • Caltech’s introductory deep learning course taught by Yasser Abu-Mostafa (YouTube)
  • Stanford CS224d: Deep Learning for Natural Language Processing (video, slides, tutorials)

abc.xyz

Google公司大规模重组:abc.xyz 。Larry Page任Alphabet公司CEO,Sergey是President,新Google新的CEO是Sundar Pichai。

如果你查看abc.xzy页面,会发现里面有向美剧《硅谷》致敬的彩蛋,hooli.xyz

hooli.xyz

 

在abc.xyz的公开信中,透露了几个重要信号

  • 大公司治理困境,员工待在舒适区太久了,就丧失了发展的斗志,公司亦然。
  • 公司强势产品会扼杀其他产品的发展壮大机会,Google准备通过拆分成独立子公司来解决。
  • 现有技术体系走到尽头,搜索技术只剩下体力活,两位创始人集中精力寻找颠覆性创新。
  • 两位创始人的Vision,无人能及。

公开信的翻译,可以看网易的中文版本。

另外,借此机会了解一下 Larry Page不为人知的故事