All Projects → NightMarcher → room-renting

NightMarcher / room-renting

Licence: other
用Python爬取安居客房源信息,并用高德地图进行可视化

Programming Languages

python
139335 projects - #7 most used programming language
HTML
75241 projects

Projects that are alternatives of or similar to room-renting

columnify
Make record oriented data to columnar format.
Stars: ✭ 28 (+75%)
Mutual labels:  bigdata
cds
Data syncing in golang for ClickHouse.
Stars: ✭ 839 (+5143.75%)
Mutual labels:  bigdata
vulkn
Love your Data. Love the Environment. Love VULKИ.
Stars: ✭ 43 (+168.75%)
Mutual labels:  bigdata
learning-spark
Tidy up Spark and Hadoop tutorials.
Stars: ✭ 28 (+75%)
Mutual labels:  bigdata
meetups-archivos
Ppts, códigos y videos de las meetups, data science days, videollamadas y workshops. Data Science Research es una organización sin fines de lucro que busca difundir, descentralizar y difundir los conocimientos en Ciencia de Datos e Inteligencia Artificial en el Perú, dando oportunidades a nuevos talentos mediante MeetUps, Workshops y Semilleros …
Stars: ✭ 60 (+275%)
Mutual labels:  bigdata
UnROOT.jl
Native Julia I/O package to work with CERN ROOT files
Stars: ✭ 52 (+225%)
Mutual labels:  bigdata
Notes
This is a learning note | Java基础,JVM,源码,大数据,面经
Stars: ✭ 69 (+331.25%)
Mutual labels:  bigdata
taller SparkR
Taller SparkR para las Jornadas de Usuarios de R
Stars: ✭ 12 (-25%)
Mutual labels:  bigdata
awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
Stars: ✭ 11,093 (+69231.25%)
Mutual labels:  bigdata
flokkr
Documentation placeholder and utilities for all the other containers.
Stars: ✭ 30 (+87.5%)
Mutual labels:  bigdata
coolplayflink
Flink: Stateful Computations over Data Streams
Stars: ✭ 14 (-12.5%)
Mutual labels:  bigdata
flink-learn
Learning Flink : Flink CEP,Flink Core,Flink SQL
Stars: ✭ 70 (+337.5%)
Mutual labels:  bigdata
Exposure
Exposure是一个帮助做曝光统计需求的库,可以很方便的对曝光事件进行埋点,在现有代码上少量侵入即可实现曝光埋点。支持RV的线性布局、网格布局、瀑布流布局、横向滑动RV,ScrollView等各种滚动布局。支持配置item的有效曝光面积。
Stars: ✭ 51 (+218.75%)
Mutual labels:  bigdata
bigdata-tech-index
Big Data Technology Index
Stars: ✭ 24 (+50%)
Mutual labels:  bigdata
SparkProgrammingInScala
Apache Spark Course Material
Stars: ✭ 57 (+256.25%)
Mutual labels:  bigdata
datacatalog-tag-manager
Python package to manage Google Cloud Data Catalog tags, loading metadata from external sources -- currently supports the CSV file format
Stars: ✭ 17 (+6.25%)
Mutual labels:  bigdata
SparkTwitterAnalysis
An Apache Spark standalone application using the Spark API in Scala. The application uses Simple Build Tool(SBT) for building the project.
Stars: ✭ 29 (+81.25%)
Mutual labels:  bigdata
ETL-Starter-Kit
📁 Extract, Transform, Load (ETL) 👷 refers to a process in database usage and especially in data warehousing. This repository contains a starter kit featuring ETL related work.
Stars: ✭ 21 (+31.25%)
Mutual labels:  bigdata
bqv
The simplest tool to manage views of BigQuery.
Stars: ✭ 22 (+37.5%)
Mutual labels:  bigdata
BigDataTools
tools for bigData
Stars: ✭ 36 (+125%)
Mutual labels:  bigdata

RoomRenting

用Python爬取安居客房源信息,并用高德地图进行可视化

脚本介绍

  • crawl_renting_info.py用来从相应租房网站上爬取租房信息,解析后存储到数据库中,与此同时将过程中的信息以日志的形式写入crawl_logger.log
  • analyse_renting_info.py用来从数据库中读取出爬取的租房信息的地址,通过调用高德地址编码API转换为经纬度,去除转换错误以及地址本身有问题的数据,最后将有效的经纬度更新到原有的数据集合中,与此同时将过程中的信息以日志的形式写入analyse_logger.log
  • export_renting_info.py用来将数据库中包含有效经纬度的租房信息以一定的格式导出到renting_info_mapdata.csv,以便之后调用高德的接口进行数据展示
  • show_renting_info.py用来对数据库中的租房信息进行按照行政区域和地铁沿线进行统计和处理,以DataFrame的形式输出,便于读者查看
  • search_renting_info.py用来与用户进行交互,通过对用户输入地址的经纬度转化,然后结合用户输入的其他租房条件,展示出距离用户输入地址一定范围的,满足用户条件的房源信息

效果展示

info_in_mongodb heat_map district_map line7 info_map_2 map_4_app sum_4_district_info sum_4_line_info

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].