【大数据和数据分析价格|大数据和数据分析报价】-猪八戒网影响国内黄金价格的多因素模型实证分析
-- 万方数据中小学数字图书馆
影响国内黄金价格的多因素模型实证分析
在经历长达20年的熊市后,中国黄金价格于2001年启动,开始了一轮长期上涨行情.然而,在连续12年的牛市大涨之后,2013年我国黄金市场终于面临首度价格大跌.究竟是什么原因影响国内黄金价格的波动?本文选取了几个可能影响黄金价格的国际和国内因素,通过Eviews6.0软件对1992年-2013年间黄金价格的季度数据进行了实证分析,研究了可能影响国内黄金价格的因素与国内金价波动之间的相关性.同时进行了模型的检验与修正,最终拟定了影响国内黄金价格的多因素模型,最后根据分析和实证检验结果提出相应的合理化政策建议.
南京师范大学商学院 江苏南京210023
Terence C.Mills.
Gold as a hedge against the US dollar.
The World Gold Council,
张志勇;翟旭.
黄金价格影响因素与中期定价模型研究.
/view/9f01dc281f155.html
2011年黄金投资展望报告.
国际黄金市场分析及我国黄金市场发展研究.
成都:四川大学,
Graham Smith.
The Price of Gold and Stock Price Indices for the United,(12).States.
The World Gold Council,
DUAN Zong-zhi
ZHANG Rong-rong
ZHOU Chong
(C) 北京万方数据股份有限公司
万方数据电子出版社房地产信贷规模影响房地产价格的研究——基于35个大中城市面扳数据分析
自1998年我国取消实物分房、实行商品房货币化以来，房地产市场经历了相对稳定发展阶段和房价快速上涨阶段，房地产业成为国民经济的重要产业。但是我国的房地产市场体系还不完善，也存在着一些问题，比如房地产供给结构不合理、一些城市房价过高等问题已经受到人们的重视和学者的关注，尤其针对房价快速上涨的因素成为研究重点。与此同时，房地产信贷规模扩张的速度也不亚于房价上涨速度，这也成为研究房价快速上涨的重要因素之一。因此本文基于年35个大中城市的面板数据，研究房地产信贷对房价的影响，并着重分析在我国不同区域、不同发展时期房地...展开
自1998年我国取消实物分房、实行商品房货币化以来，房地产市场经历了相对稳定发展阶段和房价快速上涨阶段，房地产业成为国民经济的重要产业。但是我国的房地产市场体系还不完善，也存在着一些问题，比如房地产供给结构不合理、一些城市房价过高等问题已经受到人们的重视和学者的关注，尤其针对房价快速上涨的因素成为研究重点。与此同时，房地产信贷规模扩张的速度也不亚于房价上涨速度，这也成为研究房价快速上涨的重要因素之一。因此本文基于年35个大中城市的面板数据，研究房地产信贷对房价的影响，并着重分析在我国不同区域、不同发展时期房地产信贷对房价影响的差异，以期为房地产业健康发展提出有益建议。　　
首先，回顾房地产信贷与房地产价格相关关系的研究文献，并分析总结。发现学者普遍认为房地产价格上涨的因素中包括房地产信贷规模扩张，但对其影响强度存在较大分歧；在分析房地产信贷与房地产价格的关系时，一般是基于货币政策影响资产价格的框架下进行，很少有文献通过分析信贷对房地产需求与供给的影响进而分析影响房价；国内大部分文献研究的是全国或省级层面，很少涉及市级层面的研究，并且对房地产信贷数据的界定存在差异。　　
然后，简要回顾了我国房地产业与房地产信贷的发展历程，并对相关特点进行分析。第三，通过定性与定量相结合，分析房地产信贷通过房地产需求与供给影响房价的机理，并在此分析基础上构架房地产信贷影响房价的理论模型。　　
最后，运用固定系数面板数据模型对我国不同区域房地产信贷影响房价进行实证分析以及运用变系数面板数据模型对我国不同发展时期房地产信贷影响房价进行实证分析。　　
基于以上研究内容，得出如下结论：　　
第一，房地产信贷通过影响房地产供给与需求两条途径影响房地产价格，并且房地产信贷有力支持了我国房地产业的发展。其中房地产开发经营过程中超过57％的资金来源于银行信贷或者与银行信贷有关联，并且在我国东、中、西部各个地区中，对东部地区房地产业的发展支持率最高，对中部地区房地产业发展的支持度最低。　　
第二，通过对35个大中城市的实证分析，房地产信贷对房价的影响是呈显著的正向关系，即房地产信贷扩张带动房价了上涨，但是这种影响效应在不同区域、不同发展时期均存在差异。　　
第三，房地产信贷弹性系数虽然较小，并且小于人均收入弹性系数，但是房地产信贷规模扩张速度远远大于人均收入增长速度，相对来说在助推房价上涨过程中信贷的作用强于人均收入，因此认为房地产信贷扩张是引起我国房地产价格快速上涨的主要因素。收起
学位授予单位
机标分类号
本文读者也读过
加载中，请稍候
．客服电话
&&8:00-11:30,13:00-17:00(工作日){"debug":false,"apiRoot":"","paySDK":"/api/js","wechatConfigAPI":"/api/wechat/jssdkconfig","name":"production","instance":"column","tokens":{"X-XSRF-TOKEN":null,"X-UDID":null,"Authorization":"oauth c3cef7c66aa9e6a1e3160e20"}}
{"database":{"Post":{"":{"contributes":[{"sourceColumn":{"lastUpdated":,"description":"基于R和Python的数据科学笔记本","permission":"COLUMN_PUBLIC","memberId":,"contributePermission":"COLUMN_PUBLIC","translatedCommentPermission":"all","canManage":true,"intro":"基于R和Python的数据科学笔记本","urlToken":"rdatamining","id":21246,"imagePath":"v2-f3a1ed45e62bac34a7c0a6dc.jpg","slug":"rdatamining","applyReason":"0","name":"Data Science with R&Python","title":"Data Science with R&Python","url":"/rdatamining","commentPermission":"COLUMN_ALL_CAN_COMMENT","canPost":true,"created":,"state":"COLUMN_NORMAL","followers":9678,"avatar":{"id":"v2-f3a1ed45e62bac34a7c0a6dc","template":"/{id}_{size}.jpg"},"activateAuthorRequested":false,"following":false,"imageUrl":"/v2-f3a1ed45e62bac34a7c0a6dc_l.jpg","articlesCount":47},"state":"accepted","targetPost":{"titleImage":"/v2-b2b8a23dcacbfa_r.jpg","lastUpdated":,"imagePath":"v2-b2b8a23dcacbfa.jpg","permission":"ARTICLE_PUBLIC","topics":[,1103],"summary":"本文来自于同学的投稿，经栏主本人排版编辑。前言我们都知道，民以食为天，物以稀为贵，作为一个资深吃货，吃好吃的时候就经常有小伙伴说“果然是价格越高，越好吃”，那真的是这样吗？ 在这篇文章里，我们主要以上海的美食为例，探寻这个问题 爬虫…","copyPermission":"ARTICLE_COPYABLE","translatedCommentPermission":"all","likes":0,"origAuthorId":0,"publishedTime":"T11:06:13+08:00","sourceUrl":"","urlToken":,"id":2200870,"withContent":false,"slug":,"bigTitleImage":false,"title":"【数据分析·实战】真的是价格越高，越好吃吗？","url":"/p/","commentPermission":"ARTICLE_ALL_CAN_COMMENT","snapshotUrl":"","created":,"comments":0,"columnId":21246,"content":"","parentId":0,"state":"ARTICLE_PUBLISHED","imageUrl":"/v2-b2b8a23dcacbfa_r.jpg","author":{"bio":"Tsinghua / Cornell / Quant / ML","isFollowing":false,"hash":"462c79c7d3eebc135d256fc","uid":139000,"isOrg":false,"slug":"wen-yi-yang-81","isFollowed":false,"description":"快雪时晴","name":"文兄","profileUrl":"/people/wen-yi-yang-81","avatar":{"id":"v2-54dd4c32e1cc7c62490ae21aea2ad251","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false},"memberId":,"excerptTitle":"","voteType":"ARTICLE_VOTE_CLEAR"},"id":534196}],"title":"【数据分析·实战】真的是价格越高，越好吃吗？","author":"wen-yi-yang-81","content":"本文来自于同学的投稿，经栏主本人排版编辑。前言我们都知道，民以食为天，物以稀为贵，作为一个资深吃货，吃好吃的时候就经常有小伙伴说“果然是价格越高，越好吃”，那真的是这样吗？ 在这篇文章里，我们主要以上海的美食为例，探寻这个问题爬虫数据获取首先，我爬取大众点评上上海美食的数据，包括，价格，口味，环境，服务 ，菜系等等。代码如下：hy1&-function(name,leftchar,rightchar){\n
left&-gregexpr(leftchar,name)\n
right&-gregexpr(rightchar,name)\n
for(i in 1:length(name)){\n
name[i]&-substring(name[i],left[[i]][1]+attr(left[[i]],\"match.length\"),right[[i]][1]-1)\n
name\n}\n\nmyheader&-c(\n
\"User-Agent\"=\"Mozilla/5.0 (W U; Windows NT 5.1; zh-CN; rv:1.9.1.6) \",\n
\"Accept\"=\"text/html,application/xhtml+xml,application/q=0.9,*/*;q=0.8\",\n
\"Accept-Language\"=\"en-us\",\n
\"Connection\"=\"keep-alive\",\n
\"Accept-Charset\"=\"GB2312,utf-8;q=0.7,*;q=0.7\"\n)\n\nlibrary(XML)\nlibrary(bitops)\nlibrary(RCurl)\n\ndate_all&-data.frame()\nfor (j in 1:50){\n
url&-paste(\"/search/category/1/10/p\",j,\"?aid=CCCCC2945809\",sep = \"\") \n
temp&-getURL(url,httpheader=myheader)#伪装报头访问浏览器\n
k&-strsplit(temp,\"\\n\")[[1]]\n
name1&-k[grep(\"data-hippo-type\",k)+1]\n
left&-gregexpr(\"&h4&\",name1)\n
right&-gregexpr(\"&/h4&\",name1)\n
for(i in 1:length(left)){\n
name1[i]&-substring(name1[i],left[[i]][1]+attr(left[[1]],\"match.length\"),right[[i]][1]-1)\n
name&-name1\n
price1&-k[grep(\"￥\",k)]\n
price&-hy1(price1,\"￥\",\"&/b&\")\n
price&-as.numeric(price)\n
taste1&-k[grep(\"comment-list\",k)+1]\n
taste&-hy1(taste1,\"&b&\",\"&/b&\")\n
taste&-as.numeric(taste)\n
environment1&-k[grep(\"comment-list\",k)+2]\n
environment&-hy1(environment1,\"&b&\",\"&/b&\")\n
environment&-as.numeric(environment)\n
service1&-k[grep(\"comment-list\",k)+3]\n
service&-hy1(service1,\"&b&\",\"&/b&\")\n
service&-as.numeric(service)\n
address1&-k[grep(\"tag-addr\",k)+1]\n
address&-hy1(address1,\"tag\\\"&\",\"&/span&\")\n
if (length(name)==length(price)&length(price)==length(taste)&length(taste)==length(environment)&length(environment)==length(service)) \n
date_0105&-data.frame(name,price,taste,environment,service,address)\n
date_all&-rbind(date_0105,date_all)\n
else { print(paste(\"can't get page\",j)) }\n}\n数据分析然后，我们利用爬取到的进行简单的数据分析：价格和评价的散点图library(ggplot2) \nhy1&-hy0106\nhy1&-hy1[(hy1$price&1000),]#剔除价格超过1000的点\nggplot(hy1,aes(x=taste,y=price))+geom_point()\n图中蓝绿色是日本菜，红色代表其他类。我们可以看到日本菜大多评价高，其评价和价格线性关系看不清楚，所以我们下面再具体对日本菜的数据进行具体分析。图中蓝绿色是日本菜，红色代表其他类。我们可以看到日本菜大多评价高，其评价和价格线性关系看不清楚，所以我们下面再具体对日本菜的数据进行具体分析。2. 日本菜的价格和评价的回归分析hy2&-hy1[hy1$col==1,]\nlm_hy&-lm(price~taste,data=hy2)\nsummary(lm_hy)\n\nCall:\nlm(formula = price ~ taste, data = hy2)\n\nResiduals:\n
Max \n-259.96 -119.45
494.87 \n\nCoefficients:\n
Estimate Std. Error t value Pr(&|t|)
\n(Intercept)
0.038 *\n---\nSignif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n\nResidual standard error: 152.2 on 123 degrees of freedom\nMultiple R-squared:
0.03454,\tAdjusted R-squared:
0.02669 \nF-statistic: 4.401 on 1 and 123 DF,
p-value: 0.03797\n可以看到，p值=0.03797 ，故而在0.05的显著性水平下，价格和评价是有相关性的且相关系数为正，基本上验证了“价格越高，评价越好”的说法。进一步的，那是不是所有菜都是 “价格越高，评价越好” 呢？ 3. 所有菜品价格和评价的回归分析lm_hy1&-lm(price~taste,data=hy1)\nsummary(lm_hy1)\n\nCall:\nlm(formula = price ~ taste, data = hy1)\n\nResiduals:\n
Max \n-177.75 -100.20
780.80 \n\nCoefficients:\n
Estimate Std. Error t value Pr(&|t|)\n(Intercept)
0.132\ntaste
0.552\n\nResidual standard error: 163.7 on 691 degrees of freedom\nMultiple R-squared:
0.0005133,\tAdjusted R-squared:
-0.0009331 \nF-statistic: 0.3549 on 1 and 691 DF,
p-value: 0.5516\n这次我们却发现p值=0.55，远远大于0.1，所以并不是所以菜都是“价格越高，评价越好”。至于原因，我个人认为日本菜的口味主要是看食材的新鲜程度，食材的品类，越新鲜越好的食材自然贵了，而其他菜系的口味可能受到其他因素的影响更多，不能直接反映在价格上，所以价格和口味评价之间并没有直接的线性关系。4. 菜系评价的箱线图最后，我们俩看看各个菜系的评价箱线图：从图表中我们主要可以得到以下结论：从图表中我们主要可以得到以下结论：人们对日本菜和火锅“爱憎分明”，日本菜和火锅平均评价很高，但是分布很分散，也就有说 确实有一部分人不爱吃日本菜和火锅。私房菜的评价非常高，说明现在特性化的美食更受欢迎。江浙菜和蟹宴评价都很低，这应该是太多商家只关注上海特产这个噱头并没有把精力花在提高美食上，最后被广大的吃货们识破了。最后想要了解关于R、Python、数据科学以及机器学习更多内容。请关注专栏：","updated":"T03:06:13.000Z","canComment":false,"commentPermission":"anyone","commentCount":15,"collapsedCount":0,"likeCount":98,"state":"published","isLiked":false,"slug":"","lastestTipjarors":[],"isTitleImageFullScreen":false,"rating":"none","titleImage":"/v2-b2b8a23dcacbfa_r.jpg","links":{"comments":"/api/posts//comments"},"reviewers":[],"topics":[{"url":"/topic/","id":"","name":"R（编程语言）"},{"url":"/topic/","id":"","name":"数据分析"},{"url":"/topic/","id":"","name":"数据挖掘"}],"adminClosedComment":false,"titleImageSize":{"width":600,"height":401},"href":"/api/posts/","excerptTitle":"","column":{"slug":"rdatamining","name":"Data Science with R&Python"},"tipjarState":"activated","tipjarTagLine":"真诚赞赏，手留余香","sourceUrl":"","pageCommentsCount":15,"tipjarorCount":0,"annotationAction":[],"hasPublishingDraft":false,"snapshotUrl":"","publishedTime":"T11:06:13+08:00","url":"/p/","lastestLikers":[{"bio":"数据/人工智能/胶片","isFollowing":false,"hash":"d2333acecd0ebc28f06d4de857ad5139","uid":48,"isOrg":false,"slug":"ningcn","isFollowed":false,"description":"人格分裂，别拆穿。","name":"赵宁博","profileUrl":"/people/ningcn","avatar":{"id":"v2-d071fabd89f82eb63d746be00f0df6d9","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false},{"bio":"数码打印","isFollowing":false,"hash":"7f61fb934df","uid":40,"isOrg":false,"slug":"kong-stanlee","isFollowed":false,"description":"只看集锦的曼联伪球迷。","name":"Kong Stanlee","profileUrl":"/people/kong-stanlee","avatar":{"id":"892a106a4","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false},{"bio":"金融","isFollowing":false,"hash":"501cc4d3ae07dabf674c","uid":84,"isOrg":false,"slug":"leo-lee-66","isFollowed":false,"description":"金融学","name":"守望麦田","profileUrl":"/people/leo-lee-66","avatar":{"id":"","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false},{"bio":null,"isFollowing":false,"hash":"ee365bdcdd1b7db05005fec0d402124b","uid":057000,"isOrg":false,"slug":"wang-qu-4-38","isFollowed":false,"description":"","name":"王曲","profileUrl":"/people/wang-qu-4-38","avatar":{"id":"d5d6f5ca2b9fcbe92dfcb54","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false},{"bio":null,"isFollowing":false,"hash":"06ec7e4ef12","uid":48,"isOrg":false,"slug":"ren-gong-zhi-neng-47","isFollowed":false,"description":"","name":"AIbrothels","profileUrl":"/people/ren-gong-zhi-neng-47","avatar":{"id":"c83d31d6b","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false}],"summary":"本文来自于同学的投稿，经栏主本人排版编辑。前言我们都知道，民以食为天，物以稀为贵，作为一个资深吃货，吃好吃的时候就经常有小伙伴说“果然是价格越高，越好吃”，那真的是这样吗？ 在这篇文章里，我们主要以上海的美食为例，探寻这个问题 爬虫…","reviewingCommentsCount":0,"meta":{"previous":{"isTitleImageFullScreen":false,"rating":"none","titleImage":"/v2-4d6a1bc8c3e_r.jpg","links":{"comments":"/api/posts//comments"},"topics":[{"url":"/topic/","id":"","name":"机器学习"},{"url":"/topic/","id":"","name":"数据挖掘"},{"url":"/topic/","id":"","name":"Python"}],"adminClosedComment":false,"href":"/api/posts/","excerptTitle":"","author":{"bio":"Tsinghua / Cornell / Quant / ML","isFollowing":false,"hash":"462c79c7d3eebc135d256fc","uid":139000,"isOrg":false,"slug":"wen-yi-yang-81","isFollowed":false,"description":"快雪时晴","name":"文兄","profileUrl":"/people/wen-yi-yang-81","avatar":{"id":"v2-54dd4c32e1cc7c62490ae21aea2ad251","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false},"column":{"slug":"rdatamining","name":"Data Science with R&Python"},"content":"初衷这篇文章主要从工程角度来总结在实际运用机器学习进行预测时，我们可以用哪些tips来提高最终的预测效果，主要分为Data Cleaning，Features Engineering， Models Training三个部分，可以帮助大家在实际的工作中取得更好的预测效果或是在kaggle的比赛里取得更好的成绩和排位。Data Cleaning1. 移除多余的duplicate features（相同或极为相似的features）2. 移除constant features（只有一个value的feature）#R里面可以使用unique()函数判断，如果返回值为1，则意味着为constant features\n3. 移除方差过小的features（方差过小意味着提供信息很有限）#R中可以使用caret包里的nearZeroVar()函数\n#Python里可以使用sklearn包里的VarianceThreshold()函数\n4. 缺失值处理：将missing value重新编为一类。#比如原本-1代表negative，1代表positive，那么missing value就可以全部标记为0\n#对于多分类的features做法也类似二分类的做法\n#对于numeric values,可以用很大或很小的值代表missing value比如-99999.\n5. 填补缺失值可以用mean，median或者most frequent value进行填补#R用Hmisc包中的impute()函数\n#Python用sklearn中的Imputer()函数\n6. 高级的缺失值填补方法利用其他column的features来填补这个column的缺失值（比如做回归）#R里面可以用mice包，有很多方法可供选择\n注意：不是任何时候填补缺失值都会对最后的模型预测效果带来正的效果，必须进行一定的检验。Features Engineering1. Data Transformationa. Scaling and Standardization#标准化，R用scale(), Python用StandardScaler()\n#注意：Tree based模型无需做标准化\nb. Responses Transformation#当responses展现skewed distribution时候用，使得residual接近normal distribution\n#可以用log(x),log(x+1),sqrt(x)等\n2. Features Encoding#把categorical features变成numeric feature\n#Label encoding：Python 用 LabelEncoder()和OneHotEncoder(), R用dummyVars()\n3. Features Extraction#主要是针对文本分析\n4. Features Selectiona. 方法很多：注：其中randomForest以及xgboost里的方法可以判断features的Importanceb. 此外，PCA等方法可以生成指定数量的新features（映射）c. 擅对features进行visualization或correlation的分析。Models Trainning 1. Mostly Used ML Models尝试多一些的模型，比如下面这些：2. 利用Grid Search进行hyper参数的选择3. 利用Cross-Validation衡量训练效果4. Ensemble Learning Methods必读下面这个文档：最后想要了解关于R、Python、数据科学以及机器学习更多内容。请关注我的专栏：， 以及关注我的知乎账号：","state":"published","sourceUrl":"","pageCommentsCount":0,"canComment":false,"snapshotUrl":"","slug":,"publishedTime":"T22:27:26+08:00","url":"/p/","title":"【机器学习】如何做出一个更好的Machine Learning预测模型","summary":"初衷这篇文章主要从工程角度来总结在实际运用机器学习进行预测时，我们可以用哪些tips来提高最终的预测效果，主要分为Data Cleaning，Features Engineering， Models Training三个部分，可以帮助大家在实际的工作中取得更好的预测效果或是在kaggle的比赛里…","reviewingCommentsCount":0,"meta":{"previous":null,"next":null},"commentPermission":"anyone","commentsCount":9,"likesCount":151},"next":{"isTitleImageFullScreen":false,"rating":"none","titleImage":"/v2-dc43dfb07_r.png","links":{"comments":"/api/posts//comments"},"topics":[{"url":"/topic/","id":"","name":"R（编程语言）"},{"url":"/topic/","id":"","name":"雾霾"},{"url":"/topic/","id":"","name":"雾霾治理"}],"adminClosedComment":false,"href":"/api/posts/","excerptTitle":"","author":{"bio":null,"isFollowing":false,"hash":"8f83adb551a","uid":92,"isOrg":false,"slug":"fang-chao-98","isFollowed":false,"description":"","name":"防潮同学","profileUrl":"/people/fang-chao-98","avatar":{"id":"7ff400f252f5ff29a72a8cab","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false},"content":"前言2013年9月，国务院专门出台治理大气污染的条例，王安顺代表北京市与中央签订责任状，立下壮士断腕的决心。“也是生死状，因为中央领导说，2017年实现不了空气治理就‘提头来见’。既是玩笑话，也说明了这句话的分量很重。”1月3日，北京市环保局发布了2016年北京市全年空气质量状况报告，“2016年我市空气质量达标天数198天，其中，一级优68天，二级良130天，达标天数较2015年增加12天;2016年共发生重污染39天，其中O3重污染1天，PM2.5重污染38天，较2015年减少7天。”网友们看到这份报告纷纷调侃，“不过是今年多刮了几天风罢了”。空气质量的改善是政府的功绩，还是“风”的垂怜呢？本文将基于天气以及空气质量数据，用R语言一探究竟！一、准备工作library(RCurl)\n#现成的数据找不到，需要用爬虫爬网页上的数据\nlibrary(XML)\n#提取网页中的表格\nlibrary(dplyr)\n#数据清理\nlibrary(ggplot2)\n#画图\nlibrary(stringr)\n#文本处理\n二、数据收集本文的空气质量数据来自，天气数据来自。时间跨度为至，总共应该是1096天，但2014年的空气质量数据有4天缺失，本文根据提供的数据予以补全（分别是、、、）。空气质量数据和天气数据的爬虫代码：#Air Quality\nairdata &- matrix(data = NA, nrow = 1, ncol = 10)\nairdata &- data.frame(airdata)\ncolnames(airdata) &- c(\"date\", \"airrank\", \"aqi\", \"aqirank\", \"pm2.5\", \"pm10\",\"no2\",\"so2\", \"co\", \"o3\")\n\nmonths &- sprintf(\"%02d\", 1:12)\nyears &- \nfor(i in years) {\n
for(j in months) {\n
print(paste(i, j, sep = \"\"))\n
url_a=\"/aqi/beijing-\"\n
url_b=\".html\"\n
url_all=paste(url_a, i, j, url_b, sep = \"\")\n
temp &- getURL(url_all, .encoding=\"GBK\")\n
temp2 &- iconv(temp,\"gb2312\",\"UTF-8\") \n
doc &- htmlParse(temp2, asText=T, encoding=\"UTF-8\")\n
tables &-readHTMLTable(doc)\n
airdatatemp &- as.data.frame(tables)\n
colnames(airdatatemp) &- c(\"date\", \"airrank\", \"aqi\", \"aqirank\", \"pm2.5\", \"pm10\",\"no2\",\"so2\", \"co\", \"o3\")\n
airdata &- rbind(airdata, airdatatemp)\n
}\n}\n\n\nairdata1 &- c(\"\", \"重度污染\", \"271\", \"182\", \"261\",\"263\",\"125\", \"118\", \"4.06\", \"10\")\nairdata2 &- c(\"\", \"重度污染\", \"270\", \"188\", \"220\",\"249\",\"93\",
\"1.99\", \"144\")\nairdata3 &- c(\"\", \"轻度污染\", \"107\", \"178\", \"64\", \"103\",\"45\",
\"1.08\", \"220\")\nairdata4 &- c(\"\", \"良\",
\"51\", \"80\", \"58\",
\"0.80\", \"93\")\n\nairdata &- rbind(airdata, airdata1, airdata2, airdata3, airdata4)\n\n#Wind\nwinddata &- matrix(data = NA, nrow = 1, ncol = 4)\nwinddata &- data.frame(winddata)\ncolnames(winddata) &- c(\"date\", \"weather\", \"temperature\",\"wind\")\n\nmonths &- sprintf(\"%02d\", 1:12)\nyears &- \nfor(i in years) {\n
for(j in months) {\n
url_a=\"/lishi/beijing/month/\"\n
url_b=\".html\"\n
url_all=paste(url_a, i, j, url_b, sep = \"\")\n
temp &- getURL(url_all, .encoding=\"GBK\")\n
temp2 &- iconv(temp,\"gb2312\",\"UTF-8\") \n
doc &- htmlParse(temp2, asText=T, encoding=\"UTF-8\")\n
tables &-readHTMLTable(doc)\n
winddatatemp &- as.data.frame(tables)\n
colnames(winddatatemp) &- c(\"date\", \"weather\", \"temperature\",\"wind\")\n
winddata &- rbind(winddata, winddatatemp)\n
}\n}\n三、数据清洗与整理#根据日期合并两组数据\nairdata$date &- as.Date(airdata$date)\nwinddata$date &- as.Date(winddata$date, \"%Y年%m月%d日\")\n\nmydata &- merge(airdata, winddata, by=\"date\")\nmydata &- na.omit(mydata)\nmydata &- mydata[-which(duplicated(mydata$date, )),]\n\nairdata$date &- as.Date(airdata$date)\nmydata[,3:10] &- as.numeric(unlist(mydata[,3:10]))\n\n\n#日期等数据整理\nmydata$weekday &- weekdays(mydata$date, abbreviate = T)\nmydata$weekday &- factor(mydata$weekday, levels = c(\"周一\",\"周二\",\"周三\",\"周四\",\"周五\",\"周六\",\"周日\"))\nmydata$month &- months(mydata$date, abbreviate = T)\nmydata$month &- factor(x = mydata$month, levels = c(\"1月\",\"2月\",\"3月\",\"4月\",\"5月\",\"6月\",\"7月\",\"8月\",\"9月\",\"10月\",\"11月\",\"12月\"))\nmydata$year &- format(mydata$date, \"%Y\")\nmydata$airrank &- factor(x = mydata$airrank, levels =c(\"优\",\"良\",\"轻度污染\",\"中度污染\",\"重度污染\",\"严重污染\"))\n\n#气温数据整理\nmydata$temp.max &- substr(mydata$temperature, start = 1, stop=str_locate(mydata$temperature, \"/\")[,1]-1)\nmydata$temp.min &- substr(mydata$temperature, start = str_locate(mydata$temperature, \"/\")[,1]+1, stop = nchar(mydata$temperature))\nmydata$temp.max &- as.numeric(substr(mydata$temp.max, start = 1, stop = nchar(mydata$temp.max)-43))\nmydata$temp.min &- as.numeric(substr(mydata$temp.min, start = 43, stop =nchar(mydata$temp.min)-1))\nmydata$temp.avg &- (mydata$temp.max + mydata$temp.min) / 2 \n\n#风速数据整理\nmydata$wind.day &- substr(mydata$wind, start = 1, stop=str_locate(mydata$wind, \"/\")[,1]-1)\nmydata$wind.night &- substr(mydata$wind, start =str_locate(mydata$wind, \"/\")[,1]+1, stop = nchar(mydata$wind))\nmydata$wind.day[which(is.na(mydata$wind.day))] &- mydata$wind[which(is.na(mydata$wind.day))]\nmydata$wind.night[which(is.na(mydata$wind.night))] &- mydata$wind[which(is.na(mydata$wind.night))]\nmydata$wind.day.direction &- str_split_fixed(mydata$wind.day, pattern = \" \", n = 2)[,1]\nmydata$wind.night.direction &- str_split_fixed(mydata$wind.night, pattern = \" \", n = 2)[,1]\nmydata$wind.day.speed &- str_split_fixed(mydata$wind.day, pattern = \" \", n = 2)[,2]\nmydata$wind.day.speed &- str_split_fixed(mydata$wind.day.speed, pattern = \"\\r\", n = 2)[,1]\nmydata$wind.day.speed[which(mydata$wind.day.speed==\"\")] &- \"≤3级\"\nmydata$wind.night.speed &- str_split_fixed(mydata$wind.night, pattern = \" \", n = 2)[,2]\nmydata$wind.night.speed &- str_split_fixed(mydata$wind.night.speed, pattern = \"\\r\", n = 2)[,1]\nmydata$wind.night.speed[which(mydata$wind.night.speed==\"\")] &- \"≤3级\"\n四、绘图与分析首先，我们根据所得到的AQI（空气质量指数）数据绘制了如下图所示的箱线图+小提琴图ggplot(data = mydata, aes(x = year, y = aqi)) + \n
geom_violin(fill=\"lightblue\") +\n
geom_boxplot(fill=\"lightgreen\",
width=.2) +\n
labs(x=\"\",y=\"\", title=\"北京AQI分布\") +\n
theme(plot.title = element_text(hjust=0.5, size = 15, face = \"bold\"))\n可以很明显地看到，年北京的空气质量整体上有所改善，AQI四分位线、平均值线均向下移动，总体的分布也逐渐向下偏。参与空气质量评价的主要污染物为细颗粒物、可吸入颗粒物、二氧化硫、二氧化氮、臭氧、一氧化碳等。其中PM2.5最受关注，秋冬时节PM2.5爆表的新闻常见于报端，根据同样的方法我们构造了PM2.5的趋势图，结果与AQI类似。ggplot(data = mydata, aes(x = year, y = pm2.5)) + \n
geom_violin(fill=\"lightblue\") +\n
geom_boxplot(fill=\"lightgreen\",
width=.2) +\n
labs(x=\"\",y=\"\", title=\"北京PM2.5分布\") +\n
theme(plot.title = element_text(hjust=0.5, size = 15, face = \"bold\"))\n那我们不禁要问了，近年来北京空气质量的改善，“风”——这一因素到底起了多大的作用呢？首先我们考察不同风速情况下的PM2.5分布，看看大风是不是真的吹走了雾霾。显然，空气污染（PM2.5&100）主要分布在风速小于等于3级的气象条件下，风力大于3级的日子空气污染较少。但如果从各风速看，白天和夜间呈现不同的特征，白天风速越大，PM2.5均值水平越低；夜间风速越大，PM2.5均值水平变化不太大（5-6级、6-7级均值水平较高，可能与样本太少有关）。这其中的道理，不太清楚。ggplot(data = mydata, aes(x = wind.day.speed, y = pm2.5)) +\n
geom_boxplot(fill=\"cornflowerblue\", col=1, na.rm = T) +\n
geom_point(position = \"jitter\", alpha=0.3, col=\"blue\", na.rm = T) +\n
labs(x=\"\",y=\"\", title=\"白天风速与PM2.5\") +\n
theme(plot.title = element_text(hjust=0.5, size = 15, face = \"bold\"))\n\nggplot(data = mydata, aes(x = wind.night.speed, y = pm2.5)) +\n
geom_boxplot(fill=\"cornflowerblue\", col=1, na.rm = T) +\n
geom_point(position = \"jitter\", alpha=0.3, col=\"blue\", na.rm = T) +\n
labs(x=\"\",y=\"\", title=\"夜间风速与PM2.5\") +\n
theme(plot.title = element_text(hjust=0.5, size = 15, face = \"bold\"))\n进一步地我们对比下不同年份各空气质量等级天数与各风速天数。可以看到北京的空气质量整体向好，且趋势较为明显：空气质量为优的天数明显增加，轻度污染的天数有所增加；良、中度污染的天数大致不变；中度、严重污染的天数则有所减少。而反观年的整体风速情况，则无法看到的较为明显的趋势，相对来说，2014年的整体风速较小，而2015年和2016年则大致相当。ggplot(data = mydata, aes(x = year, fill=airrank)) +\n
geom_bar(col=1) +\n
scale_fill_brewer(palette = \"YlOrRd\") +\n
labs(x=\"\",y=\"\", title=\"空气质量\") +\n
theme(plot.title = element_text(hjust=0.5, size = 15, face = \"bold\")) \n\nggplot(data = mydata, aes(x = year, fill=wind.day.speed)) +\n
geom_bar(col=1, width = 0.4) +\n
scale_fill_brewer() +\n
labs(x=\"\",y=\"\", title=\"白天风速\") +\n
theme(plot.title = element_text(hjust=0.5, size = 15, face = \"bold\")) \n\nggplot(data = mydata, aes(x = year, fill=wind.night.speed)) +\n
geom_bar(col=1, width = 0.4) +\n
scale_fill_brewer() +\n
labs(x=\"\",y=\"\", title=\"夜间风速\") +\n
theme(plot.title = element_text(hjust=0.5, size = 15, face = \"bold\"))\n综合本文的分析来看，年北京的空气质量确实是在逐渐改善，而且这其中“风”并没有扮演十分重要的角色，所以呢...","state":"published","sourceUrl":"","pageCommentsCount":0,"canComment":false,"snapshotUrl":"","slug":,"publishedTime":"T21:37:12+08:00","url":"/p/","title":"【数据分析·实战】北京的雾霾是大风吹走的吗","summary":"前言2013年9月，国务院专门出台治理大气污染的条例，王安顺代表北京市与中央签订责任状，立下壮士断腕的决心。“也是生死状，因为中央领导说，2017年实现不了空气治理就‘提头来见’。既是玩笑话，也说明了这句话的分量很重。” 1月3日，北京市环保局发布了…","reviewingCommentsCount":0,"meta":{"previous":null,"next":null},"commentPermission":"anyone","commentsCount":34,"likesCount":82}},"annotationDetail":null,"commentsCount":15,"likesCount":98,"FULLINFO":true}},"User":{"wen-yi-yang-81":{"isFollowed":false,"name":"文兄","headline":"快雪时晴","avatarUrl":"/v2-54dd4c32e1cc7c62490ae21aea2ad251_s.jpg","isFollowing":false,"type":"people","slug":"wen-yi-yang-81","bio":"Tsinghua / Cornell / Quant / ML","hash":"462c79c7d3eebc135d256fc","uid":139000,"isOrg":false,"description":"快雪时晴","profileUrl":"/people/wen-yi-yang-81","avatar":{"id":"v2-54dd4c32e1cc7c62490ae21aea2ad251","template":"/{id}_{size}.jpg"},"isOrgWhiteList":false,"badge":{"identity":null,"bestAnswerer":null}}},"Comment":{},"favlists":{}},"me":{},"global":{},"columns":{"next":{},"rdatamining":{"following":false,"canManage":false,"href":"/api/columns/rdatamining","name":"Data Science with R&Python","creator":{"slug":"wen-yi-yang-81"},"url":"/rdatamining","slug":"rdatamining","avatar":{"id":"v2-f3a1ed45e62bac34a7c0a6dc","template":"/{id}_{size}.jpg"}}},"columnPosts":{},"columnSettings":{"colomnAuthor":[],"uploadAvatarDetails":"","contributeRequests":[],"contributeRequestsTotalCount":0,"inviteAuthor":""},"postComments":{},"postReviewComments":{"comments":[],"newComments":[],"hasMore":true},"favlistsByUser":{},"favlistRelations":{},"promotions":{},"switches":{"couldAddVideo":false},"draft":{"titleImage":"","titleImageSize":{},"isTitleImageFullScreen":false,"canTitleImageFullScreen":false,"title":"","titleImageUploading":false,"error":"","content":"","draftLoading":false,"globalLoading":false,"pendingVideo":{"resource":null,"error":null}},"drafts":{"draftsList":[],"next":{}},"config":{"userNotBindPhoneTipString":{}},"recommendPosts":{"articleRecommendations":[],"columnRecommendations":[]},"env":{"isAppView":false,"appViewConfig":{"content_padding_top":128,"content_padding_bottom":56,"content_padding_left":16,"content_padding_right":16,"title_font_size":22,"body_font_size":16,"is_dark_theme":false,"can_auto_load_image":true,"app_info":"OS=iOS"},"isApp":false},"sys":{}}}