<td id="aqqih"></td>

<dd id="aqqih"></dd>
  1. <span id="aqqih"></span>
  2. <ol id="aqqih"></ol>

    nginx內存池如何實現

    這篇文章主要講解了“nginx內存池如何實現”,文中的講解內容簡單清晰,易于學習與理解,下面請大家跟著小編的思路慢慢深入,一起來研究和學習“nginx內存池如何實現”吧!

    一、簡介

    最新穩定版本nginx1.20.2。
    為了能高效、快速的分配內存,以及減少內存碎片等,nginx實現了自己的內存池基礎組件。
    主要實現文件ngx_palloc.h, ngx_palloc.c

    二、數據結構

    2.1 內存池主要結構

    typedef?struct?{
    ????u_char???????????????*last;
    ????u_char???????????????*end;
    ????ngx_pool_t???????????*next;
    ????ngx_uint_t????????????failed;
    }?ngx_pool_data_t;
    struct?ngx_pool_s?{
    ????ngx_pool_data_t???????d;
    ????size_t????????????????max;
    ????ngx_pool_t???????????*current;
    ????ngx_chain_t??????????*chain;
    ????ngx_pool_large_t?????*large;
    ????ngx_pool_cleanup_t???*cleanup;
    ????ngx_log_t????????????*log;
    };

    內存池中第一個成員是一個結構體:
    使用ngx_pool_data_t結構體來表示當前內存池信息。
    last :下次開始分配的地址
    end: 內存池的結束地址
    next: 內存池鏈表,將多個內存池連接起來

    max
    整個內存池的最大大小

    current
    指向從當前內存池開始查找可用內存

    chain
    buffer使用的,這里不涉及

    large
    當需要的內存大于內存池最大大小時,需要通過malloc直接分配,然后形成鏈表進行組織

    cleanup
    清理工作的回調鏈表

    log
    日志句柄

    2.2 大內存鏈

    當需要分配的內存比內存池的最大大小都大時,內存池無法滿足分配,所以直接從系統中分配,然后構成一個鏈表進行維護。

    typedef?struct?ngx_pool_large_s??ngx_pool_large_t;
    struct?ngx_pool_large_s?{
    ????ngx_pool_large_t?????*next;
    ????void?????????????????*alloc;
    };

    2.3 清理任務鏈

    有一個回調任務的鏈表,當內存池銷毀時,將依次遍歷此鏈表,逐一回調handler進行清理工作。

    typedef?void?(*ngx_pool_cleanup_pt)(void?*data);
    typedef?struct?ngx_pool_cleanup_s??ngx_pool_cleanup_t;
    struct?ngx_pool_cleanup_s?{
    ????ngx_pool_cleanup_pt???handler;
    ????void?????????????????*data;
    ????ngx_pool_cleanup_t???*next;
    };

    三、內存結構圖

    3.1 邏輯

    nginx內存池如何實現  nginx 第1張

    3.2 實際

    nginx內存池如何實現  nginx 第2張

    可以看出,很多節點都是從內存池中分配的,所以可以把精力都放在實際的數據上而不必在意其他細節上。

    四、實現

    4.1 創建內存池

    /*
    ?*?NGX_MAX_ALLOC_FROM_POOL?should?be?(ngx_pagesize?-?1),?i.e.?4095?on?x86.
    ?*?On?Windows?NT?it?decreases?a?number?of?locked?pages?in?a?kernel.
    ?*/
    #define?NGX_MAX_ALLOC_FROM_POOL??(ngx_pagesize?-?1)
    #define?NGX_DEFAULT_POOL_SIZE????(16?*?1024)
    ngx_pool_t?*
    ngx_create_pool(size_t?size,?ngx_log_t?*log)
    {
    ????ngx_pool_t??*p;
    ????p?=?ngx_memalign(NGX_POOL_ALIGNMENT,?size,?log);
    ????if?(p?==?NULL)?{
    ????????return?NULL;
    ????}
    ????p->d.last?=?(u_char?*)?p?+?sizeof(ngx_pool_t);
    ????p->d.end?=?(u_char?*)?p?+?size;
    ????p->d.next?=?NULL;
    ????p->d.failed?=?0;
    ????size?=?size?-?sizeof(ngx_pool_t);
    ????p->max?=?(size?<?NGX_MAX_ALLOC_FROM_POOL)???size?:?NGX_MAX_ALLOC_FROM_POOL;
    ????p->current?=?p;
    ????p->chain?=?NULL;
    ????p->large?=?NULL;
    ????p->cleanup?=?NULL;
    ????p->log?=?log;
    ????return?p;
    }

    從代碼中可以看到,內存池最大不超過pagesize的大小

    nginx內存池如何實現  nginx 第3張

    4.2 從內存池中分配空間

    分配函數分了內存對齊和內存不對齊,但這只控制了內存池中分配空間,不控制大內存分配。

    (1)分配小空間

    • 內存對齊ngx_palloc

    • 內存不對齊ngx_pnalloc

    void?*
    ngx_palloc(ngx_pool_t?*pool,?size_t?size)
    {
    #if?!(NGX_DEBUG_PALLOC)
    ????if?(size?<=?pool->max)?{
    ????????return?ngx_palloc_small(pool,?size,?1);
    ????}
    #endif
    ????return?ngx_palloc_large(pool,?size);
    }

    當需要分配的空間小于max時,將使用小內存分配方式(即從內存池中分配空間),而ngx_pnalloc和ngx_palloc相比只是調用ngx_palloc_small時的最后一個參數為0。

    從pool->current指向的內存池開始遍歷,尋找滿足分配大小的空間,找到則返回首地址

    static?ngx_inline?void?*
    ngx_palloc_small(ngx_pool_t?*pool,?size_t?size,?ngx_uint_t?align)
    {
    ????u_char??????*m;
    ????ngx_pool_t??*p;
    ????p?=?pool->current;
    ????do?{
    ????????m?=?p->d.last;
    ????????if?(align)?{
    ????????????m?=?ngx_align_ptr(m,?NGX_ALIGNMENT);
    ????????}
    ????????if?((size_t)?(p->d.end?-?m)?>=?size)?{
    ????????????p->d.last?=?m?+?size;
    ????????????return?m;
    ????????}
    ????????p?=?p->d.next;
    ????}?while?(p);
    ????return?ngx_palloc_block(pool,?size);
    }

    當現有內存池中都無法滿足分配條件時,創建新的內存池

    static?void?*
    ngx_palloc_block(ngx_pool_t?*pool,?size_t?size)
    {
    ????u_char??????*m;
    ????size_t???????psize;
    ????ngx_pool_t??*p,?*new;
    ????psize?=?(size_t)?(pool->d.end?-?(u_char?*)?pool);
    ????m?=?ngx_memalign(NGX_POOL_ALIGNMENT,?psize,?pool->log);
    ????if?(m?==?NULL)?{
    ????????return?NULL;
    ????}
    ????new?=?(ngx_pool_t?*)?m;
    ????new->d.end?=?m?+?psize;
    ????new->d.next?=?NULL;
    ????new->d.failed?=?0;
    ????m?+=?sizeof(ngx_pool_data_t);
    ????m?=?ngx_align_ptr(m,?NGX_ALIGNMENT);
    ????new->d.last?=?m?+?size;
    ????for?(p?=?pool->current;?p->d.next;?p?=?p->d.next)?{
    ????????if?(p->d.failed++?>?4)?{
    ????????????pool->current?=?p->d.next;
    ????????}
    ????}
    ????p->d.next?=?new;
    ????return?m;
    }

    其中,創建好新的內存池后,又做了一次遍歷,將failed計數加一,當大于4時,將跳過此內存池,下次就不從它開始查找。
    即認為超過4次你都不能滿足分配,以后都不能滿足分配,不再用你了,減少遍歷個數,加快成功分配效率

    (2)分配大空間

    static?void?*
    ngx_palloc_large(ngx_pool_t?*pool,?size_t?size)
    {
    ????void??????????????*p;
    ????ngx_uint_t?????????n;
    ????ngx_pool_large_t??*large;
    ????p?=?ngx_alloc(size,?pool->log);
    ????if?(p?==?NULL)?{
    ????????return?NULL;
    ????}
    ????n?=?0;
    ????for?(large?=?pool->large;?large;?large?=?large->next)?{
    ????????if?(large->alloc?==?NULL)?{
    ????????????large->alloc?=?p;
    ????????????return?p;
    ????????}
    ????????if?(n++?>?3)?{
    ????????????break;
    ????????}
    ????}
    ????large?=?ngx_palloc_small(pool,?sizeof(ngx_pool_large_t),?1);
    ????if?(large?==?NULL)?{
    ????????ngx_free(p);
    ????????return?NULL;
    ????}
    ????large->alloc?=?p;
    ????large->next?=?pool->large;
    ????pool->large?=?large;
    ????return?p;
    }

    可以看出,為了避免分配空間,遍歷large鏈查找可重用的節點,但是如果鏈表過大又可能太慢,所以只查找前三個,如果三個都沒有找到,則直接分配(而且節點也是從內存池中分配的,所以后續清理時,不需要管節點,只需要釋放申請的大內存本身)

    內存對齊

    void?*
    ngx_pmemalign(ngx_pool_t?*pool,?size_t?size,?size_t?alignment)
    {
    ????void??????????????*p;
    ????ngx_pool_large_t??*large;
    ????p?=?ngx_memalign(alignment,?size,?pool->log);
    ????if?(p?==?NULL)?{
    ????????return?NULL;
    ????}
    ????large?=?ngx_palloc_small(pool,?sizeof(ngx_pool_large_t),?1);
    ????if?(large?==?NULL)?{
    ????????ngx_free(p);
    ????????return?NULL;
    ????}
    ????large->alloc?=?p;
    ????large->next?=?pool->large;
    ????pool->large?=?large;
    ????return?p;
    }

    4.3 注冊清理任務

    ngx_pool_cleanup_t?*
    ngx_pool_cleanup_add(ngx_pool_t?*p,?size_t?size)
    {
    ????ngx_pool_cleanup_t??*c;
    ????c?=?ngx_palloc(p,?sizeof(ngx_pool_cleanup_t));
    ????if?(c?==?NULL)?{
    ????????return?NULL;
    ????}
    ????if?(size)?{
    ????????c->data?=?ngx_palloc(p,?size);
    ????????if?(c->data?==?NULL)?{
    ????????????return?NULL;
    ????????}
    ????}?else?{
    ????????c->data?=?NULL;
    ????}
    ????c->handler?=?NULL;
    ????c->next?=?p->cleanup;
    ????p->cleanup?=?c;
    ????ngx_log_debug1(NGX_LOG_DEBUG_ALLOC,?p->log,?0,?"add?cleanup:?%p",?c);
    ????return?c;
    }

    可以看出,這里只是分配了一個節點,并沒有設置handler以及data數據,所以還得看具體的調用方進行設置,因為這里返回了分配的節點。

    比如在函數ngx_create_temp_file

    ngx_int_t
    ngx_create_temp_file(ngx_file_t?*file,?ngx_path_t?*path,?ngx_pool_t?*pool,
    ????ngx_uint_t?persistent,?ngx_uint_t?clean,?ngx_uint_t?access)
    {
    ????...
    ????cln?=?ngx_pool_cleanup_add(pool,?sizeof(ngx_pool_cleanup_file_t));
    ????if?(cln?==?NULL)?{
    ????????return?NGX_ERROR;
    ????}
    ???????...
    ????????file->fd?=?ngx_open_tempfile(file->name.data,?persistent,?access);
    ...
    ????????if?(file->fd?!=?NGX_INVALID_FILE)?{
    ????????????cln->handler?=?clean???ngx_pool_delete_file?:?ngx_pool_cleanup_file;
    ????????????clnf?=?cln->data;
    ????????????clnf->fd?=?file->fd;
    ????????????clnf->name?=?file->name.data;
    ????????????clnf->log?=?pool->log;
    ????????????return?NGX_OK;
    ????????}
    ...
    }

    生成臨時文件,將fd以及文件名注冊到清理任務中,后續文件不使用了則不需要特殊處理,內存內存池釋放時將統一清理。

    4.4 重置內存池

    • 釋放大內存

    • 重置內存中last

    • 重置failed計數

    void
    ngx_reset_pool(ngx_pool_t?*pool)
    {
    ????ngx_pool_t????????*p;
    ????ngx_pool_large_t??*l;
    ????for?(l?=?pool->large;?l;?l?=?l->next)?{
    ????????if?(l->alloc)?{
    ????????????ngx_free(l->alloc);
    ????????}
    ????}
    ????for?(p?=?pool;?p;?p?=?p->d.next)?{
    ????????p->d.last?=?(u_char?*)?p?+?sizeof(ngx_pool_t);
    ????????p->d.failed?=?0;
    ????}
    ????pool->current?=?pool;
    ????pool->chain?=?NULL;
    ????pool->large?=?NULL;
    }

    這里有個現象:
    在內存池中空間不足時,將調用ngx_palloc_block創建一個新的內存池,而last指向的是m += sizeof(ngx_pool_data_t);, 因此當前新分配的內存池將比第一個內存池可用大小多了(max,current,chain,large,cleanup,log)這幾個字段大?。赡軟]有那么多,因為要對齊,可能對齊后就完全一樣了),而現在重置時,p->d.last = (u_char *) p + sizeof(ngx_pool_t);每個內存池可用大小又變成一樣的。

    4.5 銷毀內存池

    • 回調清理任務

    • 釋放大內存

    • 釋放內存池本身

    void
    ngx_destroy_pool(ngx_pool_t?*pool)
    {
    ????ngx_pool_t??????????*p,?*n;
    ????ngx_pool_large_t????*l;
    ????ngx_pool_cleanup_t??*c;
    ????for?(c?=?pool->cleanup;?c;?c?=?c->next)?{
    ????????if?(c->handler)?{
    ????????????ngx_log_debug1(NGX_LOG_DEBUG_ALLOC,?pool->log,?0,
    ???????????????????????????"run?cleanup:?%p",?c);
    ????????????c->handler(c->data);
    ????????}
    ????}
    ????for?(l?=?pool->large;?l;?l?=?l->next)?{
    ????????if?(l->alloc)?{
    ????????????ngx_free(l->alloc);
    ????????}
    ????}
    ????for?(p?=?pool,?n?=?pool->d.next;?/*?void?*/;?p?=?n,?n?=?n->d.next)?{
    ????????ngx_free(p);
    ????????if?(n?==?NULL)?{
    ????????????break;
    ????????}
    ????}
    }

    4.6 大內存釋放

    通過遍歷找到要釋放的節點,將內存釋放,并且將alloc設置成NULL,則有了節點重用的情況。

    ngx_int_t
    ngx_pfree(ngx_pool_t?*pool,?void?*p)
    {
    ????ngx_pool_large_t??*l;
    ????for?(l?=?pool->large;?l;?l?=?l->next)?{
    ????????if?(p?==?l->alloc)?{
    ????????????ngx_log_debug1(NGX_LOG_DEBUG_ALLOC,?pool->log,?0,
    ???????????????????????????"free:?%p",?l->alloc);
    ????????????ngx_free(l->alloc);
    ????????????l->alloc?=?NULL;
    ????????????return?NGX_OK;
    ????????}
    ????}
    ????return?NGX_DECLINED;
    }

    4.7 分配并清空數據

    void?*
    ngx_pcalloc(ngx_pool_t?*pool,?size_t?size)
    {
    ????void?*p;
    ????p?=?ngx_palloc(pool,?size);
    ????if?(p)?{
    ????????ngx_memzero(p,?size);
    ????}
    ????return?p;
    }

    正常分配的空間中都是垃圾數據,所以當前函數在分配空間后,將分配的空間清零。

    4.8 回調文件清理

    (1) 手動關閉指定fd

    遍歷清理任務,找到ngx_pool_cleanup_file的handler,如果是要關閉的fd,則回調

    void
    ngx_pool_run_cleanup_file(ngx_pool_t?*p,?ngx_fd_t?fd)
    {
    ????ngx_pool_cleanup_t???????*c;
    ????ngx_pool_cleanup_file_t??*cf;
    ????for?(c?=?p->cleanup;?c;?c?=?c->next)?{
    ????????if?(c->handler?==?ngx_pool_cleanup_file)?{
    ????????????cf?=?c->data;
    ????????????if?(cf->fd?==?fd)?{
    ????????????????c->handler(cf);
    ????????????????c->handler?=?NULL;
    ????????????????return;
    ????????????}
    ????????}
    ????}
    }

    (2) 關閉fd

    void
    ngx_pool_cleanup_file(void?*data)
    {
    ????ngx_pool_cleanup_file_t??*c?=?data;
    ????ngx_log_debug1(NGX_LOG_DEBUG_ALLOC,?c->log,?0,?"file?cleanup:?fd:%d",
    ???????????????????c->fd);
    ????if?(ngx_close_file(c->fd)?==?NGX_FILE_ERROR)?{
    ????????ngx_log_error(NGX_LOG_ALERT,?c->log,?ngx_errno,
    ??????????????????????ngx_close_file_n?"?\"%s\"?failed",?c->name);
    ????}
    }

    (3) 刪除文件并關閉fd

    void
    ngx_pool_delete_file(void?*data)
    {
    ????ngx_pool_cleanup_file_t??*c?=?data;
    ????ngx_err_t??err;
    ????ngx_log_debug2(NGX_LOG_DEBUG_ALLOC,?c->log,?0,?"file?cleanup:?fd:%d?%s",
    ???????????????????c->fd,?c->name);
    ????if?(ngx_delete_file(c->name)?==?NGX_FILE_ERROR)?{
    ????????err?=?ngx_errno;
    ????????if?(err?!=?NGX_ENOENT)?{
    ????????????ngx_log_error(NGX_LOG_CRIT,?c->log,?err,
    ??????????????????????????ngx_delete_file_n?"?\"%s\"?failed",?c->name);
    ????????}
    ????}
    ????if?(ngx_close_file(c->fd)?==?NGX_FILE_ERROR)?{
    ????????ngx_log_error(NGX_LOG_ALERT,?c->log,?ngx_errno,
    ??????????????????????ngx_close_file_n?"?\"%s\"?failed",?c->name);
    ????}
    }

    感謝各位的閱讀,以上就是“nginx內存池如何實現”的內容了,經過本文的學習后,相信大家對nginx內存池如何實現這一問題有了更深刻的體會,具體使用情況還需要大家實踐驗證。這里是蝸牛博客,小編將為大家推送更多相關知識點的文章,歡迎關注!

    免責聲明:本站發布的內容(圖片、視頻和文字)以原創、轉載和分享為主,文章觀點不代表本網站立場,如果涉及侵權請聯系站長郵箱:niceseo99@gmail.com進行舉報,并提供相關證據,一經查實,將立刻刪除涉嫌侵權內容。

    評論

    日本韩欧美一级A片在线观看
    <td id="aqqih"></td>

    <dd id="aqqih"></dd>
    1. <span id="aqqih"></span>
    2. <ol id="aqqih"></ol>