GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → Hyhyhyhyhyhyh → Django-Data-quality-system

Hyhyhyhyhyhyh / Django-Data-quality-system

Licence: MIT License
数据治理、数据质量检核/监控平台（Django+jQuery+MySQL）

Programming Languages

python
139335 projects - #7 most used programming language

Labels

data-quality-checksdata-qualitydata-quality-monitoringdata-quality-monitor

Projects that are alternatives of or similar to Django-Data-quality-system

datatile
A library for managing, validating, summarizing, and visualizing data.
Stars: ✭ 419 (+193.01%)
Mutual labels:  data-quality-checks, data-quality, data-quality-monitoring
re-data
re_data - fix data issues before your users & CEO would discover them 😊
Stars: ✭ 955 (+567.83%)
Mutual labels:  data-quality-checks, data-quality, data-quality-monitoring
NBi
NBi is a testing framework (add-on to NUnit) for Business Intelligence and Data Access. The main goal of this framework is to let users create tests with a declarative approach based on an Xml syntax. By the means of NBi, you don't need to develop C# or Java code to specify your tests! Either, you don't need Visual Studio or Eclipse to compile y…
Stars: ✭ 102 (-28.67%)
Mutual labels:  data-quality-checks, data-quality
hooqu
hooqu is a library built on top of Pandas-like Dataframes for defining "unit tests for data". This is a spiritual port of Apache Deequ to Python
Stars: ✭ 17 (-88.11%)
Mutual labels:  data-quality-checks, data-quality
Data-Quality-Analysis
The PEDSnet Data Quality Assessment Toolkit (OMOP CDM)
Stars: ✭ 19 (-86.71%)
Mutual labels:  data-quality-checks, data-quality
penguin-datalayer-collect
A data layer quality monitoring and validation module, this solution is part of the Raft Suite ecosystem.
Stars: ✭ 19 (-86.71%)
Mutual labels:  data-quality, data-quality-monitoring
soda-spark
Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes
Stars: ✭ 58 (-59.44%)
Mutual labels:  data-quality
osm-data-classification
Migrated to: https://gitlab.com/Oslandia/osm-data-classification
Stars: ✭ 23 (-83.92%)
Mutual labels:  data-quality
ohsome-quality-analyst
Data quality estimations for OpenStreetMap
Stars: ✭ 28 (-80.42%)
Mutual labels:  data-quality
versatile-data-kit
Versatile Data Kit (VDK) is an open source framework that enables anybody with basic SQL or Python knowledge to create their own data pipelines.
Stars: ✭ 144 (+0.7%)
Mutual labels:  data-quality
Applied Ml
📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
Stars: ✭ 17,824 (+12364.34%)
Mutual labels:  data-quality
pyrad
Python Radar Data Processing
Stars: ✭ 42 (-70.63%)
Mutual labels:  data-quality-monitoring
contessa
Easy way to define, execute and store quality rules for your data.
Stars: ✭ 17 (-88.11%)
Mutual labels:  data-quality
check-engine
Data validation library for PySpark 3.0.0
Stars: ✭ 29 (-79.72%)
Mutual labels:  data-quality
TracIn
Implementation of Estimating Training Data Influence by Tracing Gradient Descent (NeurIPS 2020)
Stars: ✭ 165 (+15.38%)
Mutual labels:  data-quality
hive compared bq
hive_compared_bq compares/validates 2 (SQL like) tables, and graphically shows the rows/columns that are different.
Stars: ✭ 27 (-81.12%)
Mutual labels:  data-quality
leila
Librería para la evaluación de calidad de datos, e interacción con el portal de datos.gov.co
Stars: ✭ 56 (-60.84%)
Mutual labels:  data-quality
great expectations action
A GitHub Action that makes it easy to use Great Expectations to validate your data pipelines in your CI workflows.
Stars: ✭ 66 (-53.85%)
Mutual labels:  data-quality
qamd
QAMyData, a data quality assurance tool for SPSS, STATA, SAS and CSV files.
Stars: ✭ 16 (-88.81%)
Mutual labels:  data-quality
DataQualityDashboard
A tool to help improve data quality standards in observational data science.
Stars: ✭ 62 (-56.64%)
Mutual labels:  data-quality
View All Similar Projects ➔

demo

http://data.sghen.cn 登录用户名密码：admin/admin

项目结构

项目
│  gconfig.py           gunicorn配置文件
│  manage.py            Django管理文件
│  README.md            readme
|  nginx.conf           nginx.conf
│
├─api                   ajax接口
│
├─authorize             身份认证模块
|
├─check                 自动检核模块
|
├─data                  仪表盘、检核明细模块
|
├─demand                更新源系统改造需求
|
├─docs                  文档目录
│
├─files                 上传下载文件模块
│
├─logs                  日志目录
|
├─mysite                Django配置目录
│
├─standard              查看、更新数据标准模块
|
├─utils                 一些复用的函数
│
└─static                css、js、附件等静态文件目录

更新记录

2020-09-05

  • 修复若干本地部署会发生的错误
  • 重要：修正部署文档docs/部署文档.md中的许多错误

2020-06-13

  • 更新血缘分析模块

2020-05

  • 数据源跟检核规则库中的数据库进行关联

2020-04-23

  1. 前端侧边栏修改，显示更加紧凑
  2. 新增数据源的查看/修改/新增功能

2020-03-29

  1. 后端
    • 检核结果由按季度存放改在按日存放，记录检核版本方便查看历史变化趋势
    • 根据check_execute_log检核日志表为前端提供日期选择接口；api代码更新为正式代码（代替随机数据）
    • 添加日期维度表
  2. 前端：在仪表盘添加各公司质量总览及全期趋势图；添加日期选择控件等
  3. 进一步前后分离，减少后端渲染模板

2019-12-29

实际部署demo

2019-09-09

demo

启停项目

# 切换虚拟环境
workon django-2.1

# 启动项目
gunicorn mysite.wsgi -c /data/pyweb/data-quality/gconfig.py &

todo

  • 数据标准编辑功能完善

说明

登录页面背景图来自https://pixabay.com

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].