深圳SEO公司
歡迎您光臨瀏覽!

SEO案例robots寫法

Robots是什么
robots可以理解為搜索引擎的協議,當蜘蛛抓取網站時第一時間是檢查網站根目錄下有沒robots,如有會優先訪問robots里的內容。
Robots 的作用是什么
robots的作用 是告訴搜索引擎哪些頁面是允許抓取和哪些頁面是不允許抓取,比如網站的數據庫,后臺,腳本等等可以禁止蜘蛛抓取。說一個高級點的知識,robots還可以用作屏蔽一些重復性頁面。
Robots寫法
User-agent: * 意思為允許或許搜索引擎抓取
Allow: 允許抓取
Disallow: 禁止抓取
*:表示全部或所有
$:結束符
#:用來注釋
舉例:
Disallow:/ 表示禁止抓取整站頁面
Allow:/ 允許抓取整站內容
常用網站程序Robots.txt寫法
織夢Robots.txt寫法
User-agent: *
Disallow: /plus/ad_js.php
Disallow: /plus/advancedsearch.php
Disallow: /plus/car.php
Disallow: /plus/carbuyaction.php
Disallow: /plus/shops_buyaction.php
Disallow: /plus/erraddsave.php
Disallow: /plus/posttocar.php
Disallow: /plus/disdls.php
Disallow: /plus/feedback_js.php
Disallow: /plus/mytag_js.php
Disallow: /plus/rss.php
Disallow: /plus/search.php
Disallow: /plus/recommend.php
Disallow: /plus/stow.php
Disallow: /plus/count.php
Disallow: /include
Disallow: /templets
Disallow: /*?*
Disallow: /*list*
Disallow: /tags.php
Disallow: /plus/flink_add.php
Disallow: /plus/flink.php
sitemap: http://www.rktpot.live/sitemap1.html
WordPress的Robots.txt寫法
User-Agent: *
Disallow: /*#comments
Disallow: /?s=
Disallow: /?author=1
Disallow: /zm/
sitemap: http://www.rktpot.live/sitemap1.html
Robots禁止抓取案例
淘寶:由于該網站的robots.txt文件存在限制指令,系統無法提供該頁面的內容描述
淘寶Robots寫法
User-agent: Baiduspider
Disallow: /
User-agent: baiduspider
Disallow: /
Robots注意事項
Disallow的寫法一定要謹慎,少了一個/會出現很多問題或多了一個*也會產生很多問題
通匹符的用法一定要正確。
本文出自深圳SEO公司,未經允許不得轉載:深圳SEO-微笑SEO服務公司 » SEO案例robots寫法
分享到: 更多 (0)

評論 3

  • 昵稱 (必填)
  • 郵箱 (必填)
  • 網址
  1. #1

    I pay a quick visit everyday some web pages and sites to read articles or reviews, except this web site presents quality based writing.

    vardenafil generic1年前 (2018-04-03)回復
  2. #2

    viagra online polskabuy generic viagra in canadareview for viagrahow does viagra work in pulmonary hypertensionjelly viagra buycomprar viagra en bogota colombiaget viagra from gpviagra online overnight deliveryviagra dosingviagra to buy ukcomprar viagra en jerezrecreational viagra use sex storieswhat is viagrapremio nobel viagrabest generic viagra canadaoda al viagra
    silagra viagra generic
    viagra online without where can i buy viagra
    buy generic viagra online uk next day delivery
    can you drink viagrahay similar de viagrapros and cons of generic viagrakan man ta 2 viagrapeople who bought viagra onlineviagra shops in singaporeviagra does it work for everyonequanto tempo demora viagra fazer efeitoviagra calciumviagra schon in jungen jahrenis cialis better than viagravirtual viagra tumblrwalgreens viagra couponschi dinh viagraviagra with prescriptionwhere can i get viagra in australiaviagra causa necrose

    Coreyson1年前 (2018-06-08)回復
  3. #3
    nygarorm3個月前 (03-24)回復
福彩3d三毛图库