How logging system Bootstrapped in Spring Boot Application
Summary Following diagram demonstrated the process to bootstrap and use Logback for loggings in Spring Boot applciation.
最近在研究CV的一些开源库(OpenCV),有一个体会就是在此领域,除了一些非常学术的_机器学习_, 深度学习_等概念外,其实还有一些很有趣的_现实的_应用场景。比如之前很流行的微软的 https://how-old.net, 你使用自己指定或者上传的照片进行面部识别_猜年龄。 如下图所示:
细想一下这个很吸引眼球的程序,其实技术本身打散了就包括两大块,一是从图片中扫描并进行面部识别,二是对找到的人脸根据算法去猜个年龄。大家可以猜猜实现第一个功能需要多少核心代码量?其实不用上万行,在这里我就使用短短几行代码(去除空格换行什么的,有效代码只要10行)就实现一个_高大上_面部识别的功能。在此文容我细述一下具体实现代码以及我对机器识别图像领域技术的理解。
_人脸识别_技术大家应该都不陌生,之前大家使用的数码相机,或者现在很多手机自带的相机都有人脸识别的功能。其效果就像是下图这样。近的看,_剁手节_刚刚过了没有多久 , 背后的马老板一直在力推的刷脸支付也是一个此领域的所谓“黑科技”。比如在德国汉诺威电子展上,马云用支付宝“刷脸”买了一套纪念邮票。人脸识别应用市场也从爆发。随后,各大互联网巨头也纷纷推出了刷脸相关的应用。
如果要加个定义,人脸识别又叫做人像识别、面部识别,是一种通过用摄像机或摄像头采集含有人脸的图像或视频流,并自动在图像中检测和跟踪人脸,进而对检测到的人脸进行脸部的一系列相关技术。
OK,长话短说,先上 干货 ,下面就是此程序的_带注释_ 版本,完整的程序以及相关配套文件可以在 这个github库 https://github.com/CloudsDocker/pyFacialRecognition 中找到,有兴趣可以_fork_ 下来玩玩。下面是整个程序的代码样子,后面我会逐行去解释分析。
就这短短的十行代码代码?seriously?“有图有真相”,我们先来看下运行的效果:
请注意 K歌二人组 的脸上的红色框框,这就是上面十行代码的成果。
因为此程序使用是的Python,因此你需要去安装Python。这里就不赘述了。除此之外,还需要安装 OpenCV (http://opencv.org/downloads.html)。 多说一句,这个 OpenCV正如其名,是一个开源的机器识别的深度学习框架。这是Intel(英特尔)实验室里的一个俄罗斯团队创造的,目前在开源社区非常的活跃。
特别提一下,对于Mac的用户,推荐使用brew去安装 (下面第一条语句可能会执行报错,我当时也是搞了好久。如果遇到第一条命令不过可以通过文尾的方式联系作者)
brew tap homebrew/science
brew install opencv
安装完成之后,在python的命令行中输入如下代码验证,如果没有报错就说明安装好了。
>>> import cv2
# -*- coding: utf-8 -*-
import cv2,sys
inputImageFile=sys.argv[1]
faceClassifier=cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
这里有几个在深度学习及机器图像识别领域中的几个概念,稍微分析一下,至于深入的知识,大家可以自行搜索或者联系作者。
在机器深度学习领域,针对识别不同物体都有不同的classifier,比如有的classifier来识别洗车,还有识别飞机的classifier,有classifier来识别照片中的笑容,眼睛等等。而我们这个例子是需要去做人脸识别,因此需要一个面部识别的classifier。
一般来说,比如想要机器学习着去识别“人脸”,就会使用大量的样本图片来事先培训,这些图片分为两大类,positive和negative的,也就是分为包“含有人脸”的图片和“不包含人脸”的图片,这样当使用程序去一张一张的分析这些图片,然后分析判断并对这些图片“分类” (classify),即合格的图片与不合格的图片,这也就其为什么叫做 classifier , 这样学习过程中积累的”知识”,比如一些判断时的到底临界值多少才能判断是positive还是negative什么的,都会存储在一个个XML文件中,这样使用这些前人经验(这里我们使用了 哈尔 分类器)来对新的图片进行‘专家判断’分析,是否是人脸或者不是人脸。
这里的 Cascade是 层级分类器 的意思。为什么要 分层 呢?刚才提到在进行机器分析照片时,其实是对整个图片从上到下,从左到右,一个像素一个像素的分析,这些分析又会涉及很多的 特征分析 ,比如对于人脸分析就包含识别眼睛,嘴巴等等,一般为了提高分析的准确度都需要有成千上万个特征,这样对于每个像素要进行成千上万的分析,对于整个图片都是百万甚至千万像素,这样总体的计算量会是个天文数字。但是,科学家很聪明,就想到分级的理念,即把这些特征分层,这样分层次去验证图片,如果前面层次的特征没有通过,对于这个图片就不用判断后面的特征了。这有点像是系统架构中的 FF (Fail Fast),这样就提高了处理的速度与效率。
objImage=cv2.imread(inputImageFile)
cvtImage=cv2.cvtColor(objImage,cv2.COLOR_BGR2GRAY)
foundFaces=faceClassifier.detectMultiScale(cvtImage,scaleFactor=1.3,minNeighbors=9,minSize=(50,50),flags = cv2.cv.CV_HAAR_SCALE_IMAGE)
print(" 在图片中找到了 {} 个人脸".format(len(foundFaces)))
for (x,y,w,h) in foundFaces:
cv2.rectangle(objImage,(x,y),(x+w,y+h),(0,0,255),2)
cv2.imshow(u'面部识别的结果已经高度框出来了。按任意键退出'.encode('gb2312'), objImage)
cv2.waitKey(0)
好了,上面是这个程序的详细解释以及相关的知识的讲解。其实这个只是个_抛砖引玉_的作用,还用非常多的应用场景,比如程序解析网页上的图片验证码,雅虎前几个月开源的 NSFW, Not Suitable for Work (NSFW),即判断那些不适合工作场所的图片,内容你懂的。 :-)
最后,再提一下,所有这些源代码及相关文件都开源在 https://github.com/CloudsDocker/pyFacialRecognition ,在fork并下载到本地后执行下面代码来测试运行
git clone https://github.com/CloudsDocker/pyFacialRecognition.git
cd pyFacialRecognition
./run.sh
如果有任何建议或者想法,请联系我。
Summary Following diagram demonstrated the process to bootstrap and use Logback for loggings in Spring Boot applciation.
Symptoms When you are using integrated authentication (Kerberos connection) for MS SqlServer connection, there is one possible error :
Why to extract resources from jar to local disk
Normal approach to debug maven
How to watch specific kubenetes deployment by labels
Background It’s typical to get various network connection issues when you run commands within corporation network. For example, you’ll find diversed issues w...
More developer friendly Threa Sleep
Summary As you know, staff and your safety is paramount. So what if emergency take place, such as fire in office, how to help yourself and your colleagues by...
Summary As you know, there are various event will be sent (multicast) when a specific story taken place.
IT-Solutions-For-Remote-Learning.md
IT-Solutions-For-Remote-Learning.md
IT-Solutions-For-Remote-Learning.md
Summary To talk to K8s for getting data, there are few approaches. While K8s’ official Java library is the most widely used one. This blog will look into thi...
Summary In windows operation system, if you want to get your CPU name, core, 64bit and speed in command line. Just follow below actions:
Summary Whitelabel Error Page is the default error page in Spring Boot web app. It provide a more user-friently error page whenever there are any issues when...
Summary I found a weird problem of the app Google Maps of my Oppo Android phone. That’s when you search a place in Google map, say “Central Park”, ideally th...
A debt security represents a debt owed by the issuer to an investor. Here, the investor acts as a lender to the issuer which may be a government, organisatio...
S3 download URL As you know, AWS S3 object can be downloaded/processed by S3 download URL. I’m showing you two examples on how to process S3 Object by NIO f...
What happened to a debug job hanging in IntelliJ (IDEAS) IDE? You may find when you try to debug a class in Intellij but it stuck there and never proceed, e....
Difference with Scala Kotlin takes the best of Java and Scala, the response times are similar as working with Java natively, which is a considerable advantag...
Shortcuts & tips
此文是作者英文原文的翻译文章,英文原文在:http://todzhang.com/posts/2018-06-10-jvm-warm-up/
Shortcuts for Slack
Gradle build stuck, keep on running but never ending
Key points of Reactive Programming
Frame in Swift
Argument Matching & Answers For example, you have mocked DOC with call(arg: Int): Intfunction. You want to return 1 if argument is greater than 5 and -1 ...
Dockers Concepts
How to decode path parameters in All REST WebServices calls
Linux Curl command
The concept of join points as matched by pointcut expressions is central to AOP, and Spring uses the AspectJ pointcut expression language by default.
As a general rule it should be possible to use the name as a pivot. Dimensions allow a particular named metric to be sliced to drill down and reason about th...
Shortcuts & tips
Here are some tips and notes about how to resolve algorithm issues listed in LeetCode Rotation problem
# Pigeonhole principle
你就会发现只要涉及递归的问题,都是 树的问题。
How to make thread-safe
A Facial Recognition utility in a dozen of python LOC (Lines Of Code)
What’s TLS TLS (Transport Layer Security) and its predecessor, SSL (Secure Sockets Layer), are security protocols designed to secure the communication betwee...
Java Deep Notes
Why JVM need warm up I don’t know how and why you get to this blog. But I know the key words in your mind are “warm” for JVM. As the name “warm up” suggested...
This is the second half about Java Concurrent of my blog
This blog is about noteworthy pivot points about Java Concurrent Framework Back to Java old days there were wait()/notify() which is error prone, while fr...
Algorithm Leetcode
Feelings is the language of the soul. If you want to know what’s true for you about something, look to how your’re feeling about.
Enable Kafka listener annotated endpoints that are created under the covers by a AbstractListenerContainerFactory. To be used on Configuration classes as fol...
Footprint
Why Terraform
Kafka
FX Spot is not covered by the regulation, as it is not considered to be a financial instrument by ESMA, the European Union (EU) regulator. As FX is considere...
currency pairs Direct ccy: means USD is part of currency pair Cross ccy: means ccy wihtout USD, so except NDF, the deal will be split to legs, both with...
nano seconds
Simple Binary Encoding (SBE)
“Cannot connect to remote desktop” with Citrix Receiver
A new type of Juice Put simply, Guice alleviates the need for factories and the use of new in your Java code. Think of Guice’s @Inject as the new new. You wi...
Key points All YAML files (regardless of their association with Ansible or not) can optionally begin with — and end with …. This is part of the YAML format a...
multithreading
Feature
What are protocol buffers?
Sudo in a Nutshell Sudo (su “do”) allows a system administrator to give certain users (or groups of users) the ability to run some (or all) commands as root...
ZK Motto the motto “ZooKeeper: Because Coordinating Distributed Systems is a Zoo.”
WHAT IS PRESTO?
Overview
Acceptance testing vs unit test It’s sometimes said that unit tests ensure you build the thing right, whereas acceptance tests ensure you build the right thi...
Scala String
philosophy The actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-o...
FileUtil.class
Camel’s message model In Camel, there are two abstractions for modeling messages, both of which we’ll cover in this section. org.apache.camel.Message—The ...
Settings
Exporting your beans to JMX The core class in Spring’s JMX framework is the MBeanExporter. This class is responsible for taking your Spring beans and registe...
Solace PubSub+ It is a message broker that lets you establish event-driven interactions between applications and microservices across hybrid cloud environmen...
Annotation retention policy What is Retention policy in java annotations?
App deployment, configuration management and orchestration - all from one system. Ansible is powerful IT automation that you can learn quickly.
Ansible: What Is It Good For? Ansible is often described as a configuration management tool, and is typically mentioned in the same breath as Chef, Puppet, a...
How Flexbox works — explained with big, colorful, animated gifs
commands:
Single Writer principle
KDB However kdb+ evaluates expressions right-to-left. There are no precedence rules. The reason commonly given for this behaviour is that it is a much simple...
Foreign Exchange markets
Better to use smart wait
Key concept In Scrum, a team is cross functional, meaning everyone is needed to take a feature from idea to implementation.
:100:DevOps Model Defined
https://stormforger.com/blog/2016/07/08/types-of-performance-testing/
Error of ‘ECONNRESET’ You may face error ECONNRESET from intranet, even appropriate proxy tools (e.g. cntlm) is running. The errors may looks like ```bash $ ...
Release & Testing Strategy There are various methods for safely releasing changes to Production. Each team must select what is appropriate for their own ...
Here is the typical erros log:
commands to read files var lineReader = require(‘readline’).createInterface({ input: require(‘fs’).createReadStream(‘C:\dev\node\input\git_reset_files.tx...
https://blog.leanstack.com/minimum-viable-product-mvp-7e280b0b9418
What is difference between declarations, providers and import in NgModule
Cross-Origin Request Sharing - CORS (A.K.A. Cross-Domain AJAX request) is an issue that most web developers might encounter, according to Same-Origin-Policy,...
Why @Effects? In a simple ngrx/store project without ngrx/effects there is really no good place to put your async calls. Suppose a user clicks on a button or...
View A view is also a responder (UIView is a subclass of UIResponder). This means that a view is subject to user interactions, such as taps and swipes. Thus,...
openshift vs openstack The shoft and direct answer is `OpenShift Origin can run on top of OpenStack. They are complementary projects that work well together....
Concepts Cloud computing is the on-demand demand delivery of compute database storage applications and other IT resources through a cloud services platform v...
whats @Effects You can almost think of your Effects as special kinds of reducer functions that are meant to be a place for you to put your async calls in suc...
The second advantage to a lazy subscription is that the observable doesn’t hold onto data by default. In the previous example, each event generated by the in...
Lettable operators RxJS 5.5, piping all the things
code E503 code E503 when run npm install packages, e.g.
The Docker project was responsible for popularizing container development in Linux systems. The original project defined a command and service (both named do...
The drawback of using Promises is that they’re unable to handle data sources that produce more than one value, like mouse movements or sequences of bytes in ...
Commands bible
Async Await keywords Async Await Support in TypeScript Async - Await has been supported by TypeScript since version 1.7. Asynchronous functions are prefixed ...
How Page Value is calculated
interface RandomAccess Marker interface used by List implementations to indicate that they support fast (generally constant time) random access. The primary ...
Secure FTP SFTP over FTP is the equivalant of HTTPS over HTTP, the security version
Setup WebSphere profiles and application in command line
After establishing a SSH session, you can install a default web server by executing sudo yum install httpd -y. To start the web server, type sudo service htt...
ORA-12899: Value Too Large for Column
Spring Bean Life Cycle Callback Methods
This is talking about Java JIT (Just-In-Time) compiler
Java Security well-behaved: programs should be prevent from consuming too much system resources
Noteworthy points about SeriableVersionUID in Java
s<-read.csv("C:/Users/xxx/dev/R/IRS/SHH_SCHISHG.csv") # aggregate s2<-table(s$Original.CP) s3<-as.data.frame(s2) # extract by Frequency ordered s3...
SFTP versus FTPS SS: Secure Shell An increasing number of our customers are looking to move away from standard FTP for transferring data, so we are ofte...
How do I remove a plug-in? Run Help > About Eclipse > Installation Details, select the software you no longer want and click Uninstall. (On Macintosh i...
Class loading subsystem
Maven philosophy “It is important to note that in the pom.xml file you specify the what and not the how. The pom.xml file can also serve as a documentatio...
Notes JDK 1.0 introduced rudimentary I/O facilities for accessing the file system (to create a directory, remove a file, or perform another task), accessi...
Net Protocols
SOA SOA is a set of design principles for building a suite of interoperable, flexible and reusable services based architecture. top-down and bottom-up a...
This page is about key points about Algorithm
Concept
What is the difference between Serializable and Externalizable in Java? In earlier version of Java, reflection was very slow, and so serializaing large ob...
What is NavigableMap
Concepts If you implement Comparable interface and override compareTo() method it must be consistent with equals() method i.e. for equal object by equals(...
Difference between equals and deepEquals of Arrays in Java Arrays.equals() method does not compare recursively if an array contains another array on oth...
Hashmap in JDK Some note worth points about hashmap Lookup process Step# 1: Quickly determine the bucket number in which this element may resid...
This blog is listing key new features introduced in Java 8
What is the difference between arbitrage and hedging?
Shortcuts Expand/collapse method body in code editor Cmd + +/- to expand and collapse a method body Show java doc Ctrl+J: To show JavaDoc
Enum Misc
verbose:gc verbose:gc prints right after each gc collection and prints details about each generation memory details. Here is blog on how to read verbose gc
contract of hashCode : Whenever it is invoked on the same object more than once during an execution of a Java application, the hashCode method must consis...
Apache
Dependency Injection Angular doesn’t automatically know how you want to create instances of your services or the injector to create your service. You must co...
ThreadLocalRandom, SecureRandm, java.util.Random, java.math.Random
JDK Versions JDK 1.5 in 2005 JDK 1.6 in 2006 JDK 1.7 in 2011 JDK 1.8 in 2014 Sun之前风光无限,但是在2010年1月27号被Oracle收购。 在被Oracle收购后对外承诺要回到每2年一个realse的节奏。但是20...
用10几行代码自己写个人脸识别程序
Eslastic Search
JSON lines
Python Scraphy
引言 有句话说有人的地方就有江湖,同样,有江湖的地方就有恩怨。在软件行业历史长河(虽然相对于其他行业来说,软件行业的历史实在太短了,但是确是充满了智慧的碰撞也是十分的精彩)中有一些恩怨情愁,分分合合的小故事,比如类似的有,从一套代码发展出来后面由于合同到期就分道扬镳,然后各自发展成独门产品的Sybase DB和微...
Hyperledger Fabric for Mortals
使用Solidity创建以太坊(Ethereum)智能合约(Smart Contract)
Reference Sublime Scope Naming Syntax Guide
大家都知道,在软件测试特别是在单元测试时,必用的一个功能就是“断言”(Assert),可能有些人觉得不就一个Assert语句,没啥花头,也有很多人用起来也是懵懵懂懂,认为只要是Assert开头的方法,拿过来就用。一个偶然的机会跟人聊到此功能,觉得还是有必要在此整理一下如何使用以及对“断言”的理解。希望可以帮助大家...
Shortcuts
深入浅出区块链系统:第一章. what you should know about blockchain
Kubernetes 和Docker Swarm 可能是使用最广泛的工具,用于在集群环境中部署容器。但是这两个工具还是有很大的差别。
在开发设计中有一些常用原则或者潜规则,根据笔者的经验,这里稍微总结一下最最常用的,以飨读者。
how to show full path in Finder window Open and run following command in terminal window defaults write com.apple.finder _FXShowPosixPathInTitle -bool true; ...
RFC origion http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.2)
The stark difference among Spark and Storm. Although both are claimed to process the streaming data in real time. But Spark processes it as micro-batches; wh...
可以想像一下,之前的传统应用系统,像是一个大办公室里面,有各个部门,销售部,采购部,财务部。办一件事情效率比较高。但是也有一些弊端,首先,各部门都在一个房间里。
What’s it Returns an unmodifiable view of the specified set. This method allows modules to provide users with “read-only” access to internal sets. Query ope...
What’s Kibana kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on...
What’s Kibana kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on...
Design philosophies
UI HTML5, AngularJS, BootStrap, REST API, JSON Backend Hadoop core (HDFS), Hive, HBase, MapReduce, Oozie, Pig, Solr
Purpose of BA 带来一些商业价值(收益) 解决业务痛点
REST API must be hypertext driver Roy’s interview
Binary Tree A binary tree is a tree in which no node can have more than two children. A property of a binary tree that is sometimes important is that th...
eBooks list of various books Node.js
Common solutions
Toggle crosshair
It’s annoying to keep on repeating typing same login and password when you access multiple systems within office or for systems in external Internet. There a...
Difference between mutal funds and hedge funds
Differences between not in, not exists , and left join with null
concepts
404 error for customized domain (such as godday) 404 There is not a GitHub Pages site here. Go to github master branch for gitpages site, manually add CN...
RQFII RQFII stands for Renminbi Qualified Foreign Institutional Investor. RQFII was introduced in 2011 to allow qualified foreign institutional investors to ...
hall-of-frame by commit numbers git shortlog -s | sort -n -r
includes() vs some()
Get permission denied error when sudo su (or hyphen in sudo command) bash: /home/YOURNAME/.bashrc: Permission denied That’s because you didn’t add “-“ hyphen...
Docker Errors
Concepts LVS means Linux Virtual Server, which is one Linux built-in component.
(‘—–Unexpected error:’, <type ‘exceptions.TypeError’>) datetime.datetime.now()
RAID RAID is Reductant Array Independent Disk,
Concepts
Description
How to setup Git in Mint Linux =================================================
DB sharding in YHD
Microservice Services are organized around capabilities, e.g., user interface front-end, recommendation, logistics, billing, etc. Services are small in ...
Codecache The maximum size of the code cache is set via the -XX:ReservedCodeCacheSize=N flag (where N is the default just mentioned for the particular com...
Script bible