xtl/python----
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
aac9f5934dfbc29881f2fe480ef2287ab18d4b6c
python----/北川/查重复学生.ipynb
T
admin aac9f5934d feat: 添加多个功能模块和工具脚本 
- 新增websocket客户端和服务端实现
- 添加图片压缩工具和快速压缩脚本
- 实现学生信息处理相关API
- 添加MQTT客户端和消息处理功能
- 更新.gitignore忽略更多文件类型
- 添加数据库操作工具和示例
- 实现多个测试脚本和工具类
2026-04-13 14:47:50 +08:00

498 lines
14 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "code",
"execution_count": 2,
"id": "cc8c5f25",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd"
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "5144b9ba",
"metadata": {},
"outputs": [],
"source": [
"df1 = pd.read_excel('./学生信息_v2筛选.xlsx')"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "425258f8",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>学号</th>\n",
" <th>姓名</th>\n",
" <th>性别</th>\n",
" <th>年级</th>\n",
" <th>班级</th>\n",
" <th>手机号</th>\n",
" <th>身份证件号</th>\n",
" <th>专业代码</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>202205270</td>\n",
" <td>王雪瑞</td>\n",
" <td>女</td>\n",
" <td>2023级</td>\n",
" <td>2322数字媒体</td>\n",
" <td>19881645679</td>\n",
" <td>510727200702202323</td>\n",
" <td>710204</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>202220063</td>\n",
" <td>秋央娜姆</td>\n",
" <td>女</td>\n",
" <td>2023级</td>\n",
" <td>2328数字媒体2班</td>\n",
" <td>13558500183</td>\n",
" <td>513332200605030824</td>\n",
" <td>710204</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>202301001</td>\n",
" <td>巴措</td>\n",
" <td>女</td>\n",
" <td>2023级</td>\n",
" <td>2301五年制电子</td>\n",
" <td>18383622297</td>\n",
" <td>513330200708083467</td>\n",
" <td>510101</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>202301002</td>\n",
" <td>白玛曲扎</td>\n",
" <td>男</td>\n",
" <td>2023级</td>\n",
" <td>2301五年制电子</td>\n",
" <td>18227480605</td>\n",
" <td>513336200805020036</td>\n",
" <td>510101</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>202301003</td>\n",
" <td>白玛曲珍</td>\n",
" <td>女</td>\n",
" <td>2023级</td>\n",
" <td>2301五年制电子</td>\n",
" <td>1552882598</td>\n",
" <td>513334200809080228</td>\n",
" <td>510101</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2977</th>\n",
" <td>5107424271057</td>\n",
" <td>丁学锋</td>\n",
" <td>男</td>\n",
" <td>2024级</td>\n",
" <td>2424运动训练</td>\n",
" <td>18281971757</td>\n",
" <td>510726200902126015</td>\n",
" <td>770303</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2978</th>\n",
" <td>5113423260389</td>\n",
" <td>张雨祺</td>\n",
" <td>女</td>\n",
" <td>2023级</td>\n",
" <td>2328数字媒体2班</td>\n",
" <td>18208164054</td>\n",
" <td>51018220071117002X</td>\n",
" <td>710204</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2979</th>\n",
" <td>5119824270112</td>\n",
" <td>王金山</td>\n",
" <td>男</td>\n",
" <td>2024级</td>\n",
" <td>2407电商升学</td>\n",
" <td>19160527010</td>\n",
" <td>510724200812220013</td>\n",
" <td>510101</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2980</th>\n",
" <td>202308640</td>\n",
" <td>刘一</td>\n",
" <td>男</td>\n",
" <td>2023级</td>\n",
" <td>2311电商军事</td>\n",
" <td>18190656121</td>\n",
" <td>510726200809206010</td>\n",
" <td>510101</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2981</th>\n",
" <td>202401540</td>\n",
" <td>董芷妍</td>\n",
" <td>女</td>\n",
" <td>2024级</td>\n",
" <td>2419数媒2班</td>\n",
" <td>17760930318</td>\n",
" <td>510726200904167021</td>\n",
" <td>710204</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>2982 rows × 8 columns</p>\n",
"</div>"
],
"text/plain": [
" 学号 姓名 性别 年级 班级 手机号 \\\n",
"0 202205270 王雪瑞 女 2023级 2322数字媒体 19881645679 \n",
"1 202220063 秋央娜姆 女 2023级 2328数字媒体2班 13558500183 \n",
"2 202301001 巴措 女 2023级 2301五年制电子 18383622297 \n",
"3 202301002 白玛曲扎 男 2023级 2301五年制电子 18227480605 \n",
"4 202301003 白玛曲珍 女 2023级 2301五年制电子 1552882598 \n",
"... ... ... .. ... ... ... \n",
"2977 5107424271057 丁学锋 男 2024级 2424运动训练 18281971757 \n",
"2978 5113423260389 张雨祺 女 2023级 2328数字媒体2班 18208164054 \n",
"2979 5119824270112 王金山 男 2024级 2407电商升学 19160527010 \n",
"2980 202308640 刘一 男 2023级 2311电商军事 18190656121 \n",
"2981 202401540 董芷妍 女 2024级 2419数媒2班 17760930318 \n",
"\n",
" 身份证件号 专业代码 \n",
"0 510727200702202323 710204 \n",
"1 513332200605030824 710204 \n",
"2 513330200708083467 510101 \n",
"3 513336200805020036 510101 \n",
"4 513334200809080228 510101 \n",
"... ... ... \n",
"2977 510726200902126015 770303 \n",
"2978 51018220071117002X 710204 \n",
"2979 510724200812220013 510101 \n",
"2980 510726200809206010 510101 \n",
"2981 510726200904167021 710204 \n",
"\n",
"[2982 rows x 8 columns]"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df1"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "3e40ca7a",
"metadata": {},
"outputs": [],
"source": [
"tem = df1[\"姓名\"].value_counts().to_dict()"
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "eaeba7cc",
"metadata": {},
"outputs": [],
"source": [
"temp = []\n",
"for i,j in tem.items():\n",
" if j>1:\n",
" temp.append(i)"
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "db508acd",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['王子轩',\n",
" '洛绒丁真',\n",
" '王俊杰',\n",
" '李鑫',\n",
" '杨杰',\n",
" '扎西邓珠',\n",
" '王鑫',\n",
" '扎西多吉',\n",
" '刘杰',\n",
" '刘欣怡',\n",
" '唐诗雨',\n",
" '刘洋',\n",
" '尼玛拉姆',\n",
" '扎西措姆',\n",
" '洛绒扎西',\n",
" '王豪',\n",
" '唐诗语',\n",
" '陈思雨',\n",
" '唐欣怡',\n",
" '张勇',\n",
" '王宏宇',\n",
" '尼玛邓珠',\n",
" '四郎',\n",
" '扎西',\n",
" '李佳鑫',\n",
" '何晓琴',\n",
" '洛绒刀登',\n",
" '赵鑫',\n",
" '李昊燃',\n",
" '李欣怡',\n",
" '卓玛拉姆',\n",
" '李佳颖',\n",
" '扎西拉姆',\n",
" '刘玉婷',\n",
" '李长平',\n",
" '王强',\n",
" '王雨辰',\n",
" '扎西泽仁',\n",
" '扎西翁姆',\n",
" '王韬',\n",
" '洛绒次称',\n",
" '罗程',\n",
" '张馨悦',\n",
" '王嘉豪',\n",
" '王志豪',\n",
" '唐浩',\n",
" '刘诗语',\n",
" '张雨婷',\n",
" '李涛',\n",
" '王诗怡',\n",
" '陈鑫',\n",
" '肖雨萱',\n",
" '登真尼玛',\n",
" '李欣蕊',\n",
" '降拥巴姆',\n",
" '李柯贤',\n",
" '扎西次仁',\n",
" '杨博文',\n",
" '泽仁拉姆',\n",
" '王欣悦',\n",
" '泽仁拥忠',\n",
" '王雪如',\n",
" '洛绒达吉',\n",
" '降央邓珠',\n",
" '杨帆',\n",
" '四郎泽仁',\n",
" '王祥',\n",
" '王昊',\n",
" '扎西巴拥',\n",
" '张洪宇',\n",
" '白玛曲珍',\n",
" '王艺',\n",
" '张浩',\n",
" '巴措',\n",
" '洛绒达洼']"
]
},
"execution_count": 31,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"temp"
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "04e4c021",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"import os"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "06c6454f",
"metadata": {},
"outputs": [],
"source": [
"temp2 = []#来自文件夹\n",
"for root, dirs, files in os.walk(\"./students/\"):\n",
" for file in files:\n",
" temp2.append(Path(file).stem)\n",
" # print(Path(file).stem)\n",
" # if Path(file).suffix in IMAGE_EXTENSIONS:\n",
" # file_path = Path(root) / file\n",
" # images.append(file_path)"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "241165d0",
"metadata": {},
"outputs": [],
"source": [
"temp3 = []\n",
"for item in temp:\n",
" if item in temp2:\n",
" temp3.append(item)\n",
" # print(item)"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "659f4fec",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[]"
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"temp3"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "9e785320",
"metadata": {},
"outputs": [],
"source": [
"import shutil"
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "a724658f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"发现重复文件: 何晓琴\n",
"发现重复文件: 刘杰\n",
"发现重复文件: 刘诗语\n",
"发现重复文件: 卓玛拉姆\n",
"发现重复文件: 唐浩\n",
"发现重复文件: 唐诗雨\n",
"发现重复文件: 四郎泽仁\n",
"发现重复文件: 巴措\n",
"发现重复文件: 张勇\n",
"发现重复文件: 张浩\n",
"发现重复文件: 张雨婷\n",
"发现重复文件: 扎西\n",
"发现重复文件: 扎西多吉\n",
"发现重复文件: 扎西巴拥\n",
"发现重复文件: 扎西次仁\n",
"发现重复文件: 扎西泽仁\n",
"发现重复文件: 扎西邓珠\n",
"发现重复文件: 李柯贤\n",
"发现重复文件: 李长平\n",
"发现重复文件: 杨帆\n",
"发现重复文件: 杨杰\n",
"发现重复文件: 泽仁拉姆\n",
"发现重复文件: 洛绒丁真\n",
"发现重复文件: 洛绒扎西\n",
"发现重复文件: 洛绒达吉\n",
"发现重复文件: 王嘉豪\n",
"发现重复文件: 王强\n",
"发现重复文件: 王志豪\n",
"发现重复文件: 王韬\n",
"发现重复文件: 登真尼玛\n",
"发现重复文件: 白玛曲珍\n",
"发现重复文件: 罗程\n",
"发现重复文件: 赵鑫\n",
"发现重复文件: 陈鑫\n",
"发现重复文件: 降央邓珠\n",
"发现重复文件: 降拥巴姆\n"
]
}
],
"source": [
"for root, dirs, files in os.walk(\"./students/\"):\n",
" for file in files:\n",
" name = Path(file).stem\n",
" if name in temp3:\n",
" print(\"发现重复文件:\", name)\n",
" shutil.move(os.path.join(root, file), os.path.join(\"./duplicates/\", file))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in New Issue View Git Blame Copy Permalink